Pith. sign in

REVIEW 3 major objections 4 minor 44 references

Optimize TSK Fuzzy Systems for Classification Problems: Mini-Batch Gradient Descent with Uniform Regularization and Batch Normalization

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A mini-batch gradient descent recipe with uniform regularization and batch normalization makes TSK fuzzy classifiers more accurate.

desk verdict A solid incremental recipe for TSK classifier training, but the abstract's integration claim is not backed by the paper's own significance tests. read the letter →

arxiv 1908.00636 v3 pith:5WAYP3FG submitted 2019-08-01 cs.LG stat.ML

Yuqi Cui , Jian Huang , Dongrui Wu This is my paper · ORCID
classification cs.LGstat.ML
keywords Takagi-Sugeno-Kangfuzzyclassifiersmini-batchgradientdescentuniformregularizationbatchnormalizationAdaBoundclassificationbenchmarkdatasets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Takagi–Sugeno–Kang (TSK) fuzzy classifiers are interpretable but have lagged on large or high-dimensional data because their usual training methods are slow or memory-hungry. This paper argues that a mini-batch gradient descent (MBGD) approach using the AdaBound optimizer, plus two additions, closes that gap for classification. The first addition, uniform regularization (UR), adds a penalty that pushes every rule toward the same average firing level, so no small set of rules dominates the output; the second, batch normalization (BN), normalizes the inputs feeding the rule consequents in each mini-batch and accelerates convergence. Across 12 public benchmark datasets, the paper reports that each technique improves accuracy on its own and that their combination achieves the best average balanced accuracy and near-best raw accuracy among nine compared classifiers.

What carries the argument

The load-bearing machinery is threefold. First, the uniform-regularization loss $\ell_{UR} = \sum_{r=1}^{R}\big(\frac{1}{N}\sum_{n=1}^{N}\bar{f}_r(x_n)-\tau\big)^2$, where $\bar{f}_r(x) = f_r(x)/\sum_i f_i(x)$ is the normalized firing level of rule $r$, is added to the cross-entropy loss with trade-off weight $\lambda$. Second, a batch-normalization (BN) layer shared by all rule consequents normalizes each mini-batch's inputs before they enter the linear consequents and is merged back into the consequent coefficients at test time. Third, AdaBound serves as the base optimizer, behaving like Adam early and like stochastic gradient descent later. The conceptual engine is the TSK/stacking-ensemble equivalence: if each rule is a base learner and its firing level is the ensemble weight, then the “rich get richer” concentration of firing on a few rules starves most base learners, and UR is designed to reverse that concentration.

What would settle it

Inspect the learned membership-function variances after training TSK-MBGD-UR on a well-separated classification problem: if most $\sigma_{r,d}$ are inflated while per-rule accuracy on localized clusters does not improve, then uniform firing is not diversifying the rules, and the accuracy gain must be attributed to something other than the stated mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that the two main obstacles to training TSK fuzzy classifiers at scale—slow optimization and under-used rules—can be removed by two modifications to mini-batch gradient descent. Uniform regularization appends to the cross-entropy loss a term that penalizes deviation of each rule's average normalized firing level from a uniform target, on the theory that a TSK system is functionally equivalent to a stacking ensemble in which each rule is a base learner and its firing level is the aggregation weight; uniform firing therefore recruits more base learners and improves generalization. Batch normalization, extended from deep networks, normalizes the inputs used to compute the rule consequents in each mini-batch, which the paper observes enlarges gradient magnitudes and speeds convergence, and at test time the normalization can be folded into the consequent layer so the original TSK architecture is preserved. The paper concludes that UR and BN are complementary: integrating them yields TSK-MBGD-UR-BN, the best performer on average balanced accuracy and the runner-up on average raw accuracy among the nine algorithms.

Load-bearing premise

That making rules fire at similar average rates improves generalization because it activates more useful base learners, rather than merely widening membership functions so every rule fires everywhere.

Editorial extensions

If this is right

  • Adding UR to the loss lowers the average rank of the MBGD-trained TSK classifiers on both raw and balanced accuracy across the 12 datasets, whether or not BN is present.
  • Adding BN alone lowers training loss faster and raises test accuracy, and its effect is visible in larger L1 norms of both antecedent and consequent gradients in early epochs.
  • UR and BN are complementary: the combined TSK-MBGD-UR-BN achieves the best average BCA rank (2.6) and the second-best average RCA rank (3.0), with statistically significant wins over CART, JRip, PART, plain TSK-MBGD, and TSK-MBGD-BN.
  • Because the BN layer folds into the consequent layer at test time, the improved classifier keeps the original TSK architecture, so the accuracy gains do not come at the cost of a harder-to-interpret model.
  • Batch sizes in the range 64–256 appear to give the best test performance for TSK-MBGD-UR-BN, with both very small and very large batches degrading accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's explanation for UR—uniform firing activates more base learners—is not directly tested; the reported firing-level and entropy changes are also consistent with membership functions simply becoming wider. A natural follow-up would measure whether rules remain locally specialized after UR.
  • The regularization target appears as $1/R$ in the loss equation but as $1/C$ in the text; comparing these two targets on imbalanced data could reveal whether UR's benefit is mainly a class-balancing effect.
  • The test-time fold-in of BN suggests a general recipe: any affine input normalization used in training can be absorbed into the linear consequents of a TSK system, so normalizers other than BN could be ported to fuzzy classifiers in the same way.
  • Because the combined classifier beats random forests on balanced accuracy but not on raw accuracy, UR+BN may act partly as an implicit remedy for class imbalance; constructing intentionally skewed datasets with different $\tau$ targets could test this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes a mini-batch gradient descent (MBGD) training algorithm for TSK fuzzy classifiers, combining AdaBound, a uniform regularization (UR) term that penalizes deviations of average normalized firing levels from a target, and batch normalization (BN) applied to rule consequents. It evaluates nine algorithms on 12 UCI datasets with 70/30 splits repeated 30 times, reporting raw and balanced classification accuracy (RCA and BCA), ranks, and Dunn's FDR-corrected multiple comparisons. The central claims are that UR and BN each improve over the MBGD baseline and that integrating them further improves classification performance.

Significance. If the results hold, the paper offers a practical and reproducible training recipe for TSK fuzzy classifiers at moderate scale. The evaluation has real strengths: 30 random splits per dataset, 12 external UCI benchmarks, rank-based analysis, FDR correction, and a public Python implementation. The individual effects of UR and BN versus the MBGD baseline are statistically significant on both accuracy measures. However, the headline integration gain is not supported by the paper's own inferential tests, and the mechanism proposed for UR remains under-verified. The contribution is therefore solid but needs a more cautious claim and additional evidence before publication.

major comments (3)
  1. [Abstract and Section III-E, Table VI] The abstract states that 'integrating them can further improve the classification performance,' and Observation 3 in Section III-D claims TSK-MBGD-UR-BN 'almost always performed the best.' However, the Dunn/FDR comparisons in Table VI show that against TSK-MBGD-UR, the combined method yields p=0.1409 (RCA) and p=0.0986 (BCA), both above 0.05. The better average ranks (3.0 RCA, 2.6 BCA) could arise from sampling noise; calling 0.0986 'quite close' to the threshold (Section III-E) is not an evidential argument. This is load-bearing because the integration benefit is a distinct component of the contribution. Please either soften the claim to 'may improve' or add evidence, for example a paired test across datasets with effect sizes or additional datasets.
  2. [Section II-B, Eqs. (7)-(8)] There is a direct inconsistency in the UR objective. Equation (7) defines ℓ_UR with a target τ, and the text says τ is set to 1/C, where C is the number of classes. Equation (8), however, writes the penalty as ((1/N)Σ_n f_r(x_n) − 1/R)^2, and Section III-F again says 'we set τ = 1/C in (8).' Since the normalized firing levels sum to 1, the uniform target is 1/R (R=20 in the experiments), not 1/C (which ranges from 2 to 10 across datasets). The correct target changes the regularizer substantially. Please clarify which expression was implemented, and if τ=1/C was used, explain how a target not summing to 1 was handled and how τ was selected.
  3. [Section III-F, Figures 5-6] The explanation that uniform firing improves generalization by activating more base learners (via the functional equivalence in [32]) is not tested against the alternative explanation that UR simply broadens membership functions. Figure 6 shows a sharp rise in per-input firing entropy under UR, but broadened Gaussians with larger σ_{r,d} would also raise entropy while destroying local specialization. The paper does not report whether rules remain locally specialized, for example the per-rule receptive-field size or the distribution of inputs on which each rule has the highest firing level. If UR mostly inflates variances, the stated mechanism fails even if the accuracy gain is real. Please add such a diagnostic or explicitly weaken the mechanistic claim.
minor comments (4)
  1. [Section II-B, Eq. (8)] The symbol N is used both for the full dataset size and for the mini-batch size in Eq. (8); please rename the batch size (e.g., B) to avoid ambiguity.
  2. [Section III-B] The description 'we randomly sampled 20% data from the training set and trained the TSK model with early stopping five times' is ambiguous; please clarify whether the 20% is a validation subset and how the final model is trained on the full training set.
  3. [Section IV] The statement that the algorithm 'may be the only algorithm that can train a TSK fuzzy classifier on big and high-dimensional datasets' is too strong, since no comparison with other scalable TSK training methods on very large data is made; please temper the claim.
  4. [Figure 7 caption] The horizontal axis starts at epoch 3, so the first two epochs are not shown; this should be explicitly stated in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims rest on held-out, external UCI evaluations, not on a fitted quantity or a self-referential derivation.

full rationale

The paper's central claims are empirical. TSK-MBGD-UR-BN is evaluated on 70/30 splits of 12 UCI datasets, with test scores computed on held-out data. UR's loss term (Eq. 8) directly penalizes deviation of mini-batch average normalized firing levels from 1/R, and the regularization weight λ is selected by cross-validation on the training set; therefore, observing more uniform firing levels in Fig. 5 is a verification of the regularizer's intended behavior, not a circular prediction. The author self-citations ([6], [32]) supply optimizer settings and a functional-equivalence theorem, neither of which is used as the evidence for the headline performance claim. The cited equivalence in [32] is a parameter-free theoretical result stated independently of the present experiments, so it qualifies as independent support under the review rules. Table VI shows p = 0.1409 (RCA) and p = 0.0986 (BCA) for TSK-MBGD-UR-BN versus TSK-MBGD-UR, meaning the abstract's 'integrating them can further improve' claim is only weakly supported statistically; however, that is an evidential weakness, not circularity. No fitted parameter is renamed as a prediction, and no derivation uses its conclusion as an input.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's contribution is empirical, so its ledger is mostly design choices: lambda tuned by cross-validation, alpha, R, and the learning rate fixed by hand, and an ambiguous UR target (1/R in Eq. (8) versus the 1/C claimed in the text). The mechanistic premises are borrowed: functional equivalence from the authors' own [32], the rich-get-richer degradation from the MoE papers [33] and [34], and BN's benefit is assumed with no theory, explicitly admitted in Section III-G. No invented entities: the UR term is a loss component, not a postulated object, and the paper introduces no new forces, particles, or latent mechanisms.

free parameters (6)
  • lambda (UR weight) = searched over {0.1, 1, 10, 20, 50} by 5-fold cross-validation
    Strength of the uniform regularization term in Eq. (8), tuned per dataset on the training set.
  • alpha (L2 weight) = 0.05
    Hand-fixed L2 regularization weight for all TSK variants.
  • R (number of rules) = 20
    Hand-fixed model capacity for all TSK variants on all datasets.
  • AdaBound learning rate = 0.01
    Borrowed from the authors' prior regression paper [6], not re-tuned.
  • UR target firing level = 1/R in Eq. (8); text claims 1/C
    Design choice for the expected rule firing level; the text/equation mismatch makes the optimized objective ambiguous.
  • Batch size = not reported for the main experiments
    Section III-H studies batch size only on Satellite; the value used for Tables II and III is not stated.
assumptions (5)
  • domain assumption TSK systems are functionally equivalent to mixture-of-experts and stacking ensembles
    Used in Sections II-B and III-F to justify why uniform rule firing should improve generalization; grounded in the authors' own paper [32].
  • domain assumption The rich-get-richer effect degrades generalization in TSK as it does in MoE
    Section II-B transfers the MoE failure mode ([33], [34]) to TSK without TSK-specific evidence before the experiments.
  • ad hoc to paper More uniformly used rules means more active base learners, hence better generalization
    Section III-F offers this as the explanation for UR's benefit; it is only tested indirectly through firing-level histograms and entropies.
  • domain assumption Batch normalization improves TSK training even without a theory of why
    Section III-G explicitly declines to give a theoretical account, citing [24]; the transfer from deep networks to TSK is assumed.
  • standard math Mini-batch statistics approximate full-dataset statistics for the UR and BN terms
    Eqs. (7)-(9) are defined on full data but applied to mini-batches; Section III-H notes small batches destabilize training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimize TSK Fuzzy Systems for Classification Problems: Mini-Batch Gradient Descent with Uniform Regularization and Batch Normalization." pith.science (2026). https://pith.science/paper/5WAYP3FG

@misc{pith2026190800636,
  author       = {Pith},
  title        = {Pith review of: Optimize TSK Fuzzy Systems for Classification Problems: Mini-Batch Gradient Descent with Uniform Regularization and Batch Normalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5WAYP3FG}},
  note         = {Machine review of arXiv:1908.00636}
}
read the original abstract

Takagi-Sugeno-Kang (TSK) fuzzy systems are flexible and interpretable machine learning models; however, they may not be easily optimized when the data size is large, and/or the data dimensionality is high. This paper proposes a mini-batch gradient descent (MBGD) based algorithm to efficiently and effectively train TSK fuzzy classifiers. It integrates two novel techniques: 1) uniform regularization (UR), which forces the rules to have similar average contributions to the output, and hence to increase the generalization performance of the TSK classifier; and, 2) batch normalization (BN), which extends BN from deep neural networks to TSK fuzzy classifiers to expedite the convergence and improve the generalization performance. Experiments on 12 UCI datasets from various application domains, with varying size and dimensionality, demonstrated that UR and BN are effective individually, and integrating them can further improve the classification performance.

Figures

Figures reproduced from arXiv: 1908.00636 by the authors.

Figure 1
Figure 1. Mixture of experts (MoE) [29]. Although MoE has been used successfully in many applica￾tions, it may suffer from the “rich get richer” effect [33], [34]: once an expert is slightly better than others, it is always picked by the gating network, whereas other experts starve and are rarely used. This is bad for the generalization performance of the overall model. Since MoE and TSK fuzzy systems are functionally equiv￾a… view at source ↗
Figure 2
Figure 2. BN in training a TSK fuzzy classifier. All rule consequ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. (a) TSK fuzzy system with global BN (TSK-MBGD-UR-GBN); and, (b) TSK fuzzy system with rule-specific BN (TSK-MBGD-UR-RBN). III. EXPERIMENTS AND RESULTS This section validates the performances of our proposed UR and BN on multiple datasets from various application domains, with varying size and feature dimensionality. A. Datasets We evaluated our proposed algorithms on 12 classification datasets from the UCI Machine L… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (a) RCAs and (b) BCAs of the four MBGD-based TSK fuzzy [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Average normalized rule firing levels of [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: (a) Training loss, (b) test BCA, (c) L1 norm of the ante [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Average RCAs and BCAs of TSK-MBGD-UR-BN on the Satellite dataset, using different batch sizes. IV. CONCLUSIONS AND FUTURE RESEARCH TSK fuzzy systems are powerful and frequently used ma￾chine learning models, for both regression and classifica￾tion. However, they may no…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 34 canonical work pages

  1. [32]

    On the Functional Equivalence of TSK Fuzzy Systems to Neural Networks, Mixture of Experts, CART, and Stacking Ensemble Regression

    D. Wu, C.-T. Lin, J. Huang, and Z. Zeng, “On the functiona l equivalence of TSK fuzzy systems to neural networks, mixture of experts, CART, and stacking ensemble regression,” IEEE Trans. on Fuzzy Systems , 2020, in press. [Online]. Available: https://arxiv.org/a bs/1903.10572

    work page Pith review arXiv 2020

  2. [1]

    Fuzzy control systems: Past, present and future ,

    A.-T. Nguyen, T. Taniguchi, L. Eciolaza, V . Campos, R. Pa lhares, and M. Sugeno, “Fuzzy control systems: Past, present and future ,” IEEE Computational Intelligence Magazine , vol. 14, no. 1, pp. 56–68, 2019

    work page 2019

  3. [2]

    Implementation of evol utionary fuzzy systems,

    Y . Shi, R. Eberhart, and Y . Chen, “Implementation of evol utionary fuzzy systems,” IEEE Trans. on Fuzzy Systems , vol. 7, no. 2, pp. 109–119, 1999

    work page 1999

  4. [3]

    Genetic learning and performance eva luation of interval type-2 fuzzy logic controllers,

    D. Wu and W. W. Tan, “Genetic learning and performance eva luation of interval type-2 fuzzy logic controllers,” Engineering Applications of Artificial Intelligence , vol. 19, no. 8, pp. 829–841, 2006

    work page 2006

  5. [4]

    Back-propagation of fuzzy s ystems as nonlinear dynamic system identifiers,

    L.-X. Wang and J. M. Mendel, “Back-propagation of fuzzy s ystems as nonlinear dynamic system identifiers,” in Proc. IEEE Int’l Conf. on Fuzzy Systems , San Diego, CA, Sep. 1992, pp. 1409–1418

    work page 1992

  6. [5]

    ANFIS: Adaptive-network-based fuzzy inf erence system,

    J. S. R. Jang, “ANFIS: Adaptive-network-based fuzzy inf erence system,” IEEE Trans. on Systems, Man, and Cybernetics , vol. 23, no. 3, pp. 665– 685, 1993

    work page 1993

  7. [6]

    Optimize TSK Fuzzy Systems for Regression Problems: Mini-Batch Gradient Descent with Regularization, DropRule and AdaBound (MBGD-RDA)

    D. Wu, Y . Y uan, J. Huang, and Y . Tan, “Optimize TSK fuzzy systems for big data regression problems: Mini-batch gradi ent descent with regularization, DropRule and AdaBound (MBGD-RDA),” IEEE Trans. on Fuzzy Systems , 2020, in press. [Online]. Available: https://arxiv.org/abs/1903.10951 10

    work page Pith review arXiv 2020

  8. [7]

    Fuzzy modeling of high-dimensional systems: co mplexity reduc- tion and interpretability improvement,

    Y . Jin, “Fuzzy modeling of high-dimensional systems: co mplexity reduc- tion and interpretability improvement,” IEEE Trans. on Fuzzy Systems , vol. 8, no. 2, pp. 212–221, 2000

    work page 2000

Show all 44 references
  1. [8]

    A hierarchica l fused fuzzy deep neural network for data classification,

    Y . Deng, Z. Ren, Y . Kong, F. Bao, and Q. Dai, “A hierarchica l fused fuzzy deep neural network for data classification,” IEEE Trans. on Fuzzy Systems, vol. 25, no. 4, pp. 1006–1012, 2016

    2016

  2. [9]

    From minimum enclosin g ball to fast fuzzy inference system training on large datasets,

    F.-L. Chung, Z. Deng, and S. Wang, “From minimum enclosin g ball to fast fuzzy inference system training on large datasets,” IEEE Trans. on Fuzzy Systems , vol. 17, no. 1, pp. 173–184, 2008

    2008

  3. [10]

    A multi- criteria collaborative filtering recommender system for th e tourism domain using Expectation Maximization (EM) and PCA–ANFIS,

    M. Nilashi, O. Bin Ibrahim, N. Ithnin, and N. H. Sarmin, “ A multi- criteria collaborative filtering recommender system for th e tourism domain using Expectation Maximization (EM) and PCA–ANFIS, ” Elec- tronic Commerce Research and Applications, vol. 14, no. 6, pp. 542–562, 2015

    2015

  4. [11]

    Fa ult diagnosis of Tennessee Eastman process with multi-scale PC A and ANFIS,

    C. K. Lau, K. Ghosh, M. A. Hussain, and C. R. C. Hassan, “Fa ult diagnosis of Tennessee Eastman process with multi-scale PC A and ANFIS,” Chemometrics and Intelligent Laboratory Systems , vol. 120, pp. 1–14, 2013

    2013

  5. [12]

    A sur vey on soft subspace clustering,

    Z. Deng, K.-S. Choi, Y . Jiang, J. Wang, and S. Wang, “A sur vey on soft subspace clustering,” Information Sciences, vol. 348, pp. 84–106, 2016

    2016

  6. [13]

    Enhanced soft subspace clustering integrating within-cluster and betwe en-cluster in- formation,

    Z. Deng, K.-S. Choi, F.-L. Chung, and S. Wang, “Enhanced soft subspace clustering integrating within-cluster and betwe en-cluster in- formation,” Pattern Recognition, vol. 43, no. 3, pp. 767–781, 2010

    2010

  7. [14]

    MET SK-HDe: A multiobjective evolutionary algorithm to learn accurate TSK-fuzzy systems in high-dimensional and large-scale regression pr oblems,

    M. J. Gacto, M. Galende, R. Alcal´ a, and F. Herrera, “MET SK-HDe: A multiobjective evolutionary algorithm to learn accurate TSK-fuzzy systems in high-dimensional and large-scale regression pr oblems,” In- formation Sciences , vol. 276, pp. 63–79, 2014

    2014

  8. [15]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, and A. Courville, Deep Learning . Boston, MA: MIT press, 2016

    2016

  9. [16]

    An overview of gradient descent optimizatio n algorithms,

    S. Ruder, “An overview of gradient descent optimizatio n algorithms,” arXiv preprint arXiv:1609.04747 , 2016

    2016 arXiv

  10. [17]

    Large-scale machine learning with stochas tic gradient de- scent,

    L. Bottou, “Large-scale machine learning with stochas tic gradient de- scent,” in Proc. Int’l Conf. on Computational Statistics . Paris, France: Springer, Aug. 2010, pp. 177–186

    2010

  11. [18]

    On the importance of initialization and momentum in deep learning,

    I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the importance of initialization and momentum in deep learning,” in Proc. Int’l Conf. on Machine Learning , Atlanta, GA, Jun. 2013, pp. 1139–1147

    2013

  12. [19]

    Adam: A method for stochastic opt imization,

    D. P . Kingma and J. Ba, “Adam: A method for stochastic opt imization,” in Proc. Int’l Conf. on Learning Representations , San Diego, CA, May 2015

    2015

  13. [20]

    The marginal value of adaptive gradient methods in machine lear ning,

    A. C. Wilson, R. Roelofs, M. Stern, N. Srebro, and B. Rech t, “The marginal value of adaptive gradient methods in machine lear ning,” in Proc. Advances in Neural Information Processing Systems , Long Beach, CA, Dec. 2017, pp. 4148–4158

    2017

  14. [21]

    Improving generalization p erformance by switching from Adam to SGD,

    N. S. Keskar and R. Socher, “Improving generalization p erformance by switching from Adam to SGD,” arXiv preprint arXiv:1712.07628 , 2017

    2017 arXiv

  15. [22]

    Adaptive gradient m ethods with dynamic bound of learning rate,

    L. Luo, Y . Xiong, Y . Liu, and X. Sun, “Adaptive gradient m ethods with dynamic bound of learning rate,” in Proc. Int’l Conf. on Learning Representations, New Orleans, LA, May 2019

    2019

  16. [23]

    Batch normalization: Acceler ating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Acceler ating deep network training by reducing internal covariate shift,” in Proc. Int’l Conf. on Machine Learning , Lille, France, Jul. 2015

    2015

  17. [24]

    How do es batch nor- malization help optimization?

    S. Santurkar, D. Tsipras, A. Ilyas, and A. Madry, “How do es batch nor- malization help optimization?” in Proc. Advances in Neural Information Processing Systems, Montral , Canada, Dec. 2018, pp. 2483–2493

    2018

  18. [25]

    Layer normalizat ion,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalizat ion,” arXiv preprint arXiv:1607.06450 , 2016

    2016 arXiv

  19. [26]

    Revisit fuzzy neural network: Demystifying ba tch normalization and ReLU with generalized hamming network,

    L. Fan, “Revisit fuzzy neural network: Demystifying ba tch normalization and ReLU with generalized hamming network,” in Proc. Advances in Neural Information Processing Systems , Long Beach, CA, Dec. 2017, pp. 1923–1932

    2017

  20. [27]

    Fas t and accurate deep network learning by exponential linear units (ELUs),

    D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fas t and accurate deep network learning by exponential linear units (ELUs),” arXiv preprint arXiv:1511.07289, 2015

    2015 arXiv

  21. [28]

    Group normalization,

    Y . Wu and K. He, “Group normalization,” in Proc. European Conf. on Computer Vision, Munich, Germany, Sep. 2018, pp. 3–19

    2018

  22. [29]

    Adaptive mixtures of local experts,

    R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural Computation, vol. 3, no. 1, pp. 79–87, 1991

    1991

  23. [30]

    Now comes the time to defuzz ify neuro- fuzzy models,

    H. Bersini and G. Bontempi, “Now comes the time to defuzz ify neuro- fuzzy models,” Fuzzy Sets and Systems , vol. 90, no. 2, pp. 161–169, 1997

    1997

  24. [31]

    Comments on ‘fun ctional equivalence between radial basis function networks and fuz zy inference systems’ [and author’s reply],

    H. Andersen, A. Lotfi, and L. Westphal, “Comments on ‘fun ctional equivalence between radial basis function networks and fuz zy inference systems’ [and author’s reply],” IEEE Trans. on Neural Networks , vol. 9, no. 6, pp. 1529–1532, 1998

    1998

  25. [33]

    Mixture models for diverse machine translation: Tricks of the trade,

    T. Shen, M. Ott, M. Auli, and M. Ranzato, “Mixture models for diverse machine translation: Tricks of the trade,” arXiv preprint arXiv:1902.07816, 2019

    1902 arXiv

  26. [34]

    Outrageously large neural networks: The spars ely-gated mixture-of-experts layer,

    N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G . Hinton, and J. Dean, “Outrageously large neural networks: The spars ely-gated mixture-of-experts layer,” arXiv preprint arXiv:1701.06538 , 2017

    2017 arXiv

  27. [35]

    Deep residual learni ng for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learni ng for image recognition,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Las V egas, NV , Jun. 2016, pp. 770–778

    2016

  28. [36]

    Wide residual networks ,

    S. Zagoruyko and N. Komodakis, “Wide residual networks ,” arXiv preprint arXiv:1605.07146 , 2016

    2016 arXiv

  29. [37]

    Densely connected convolutional networks,

    G. Huang, Z. Liu, L. V an Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, Jul. 2017, pp. 4700–4708

    2017

  30. [38]

    Generating accurate rule set s without global optimization,

    E. Frank and I. H. Witten, “Generating accurate rule set s without global optimization,” in Proc. Int’l Conf. on Machine Learning , San Francisco, CA, Jul. 1998

    1998

  31. [39]

    Repeated incremental pruning to produce e rror reduc- tion,

    W. W. Cohen, “Repeated incremental pruning to produce e rror reduc- tion,” in Proc. Int’l Conf. on Machine Learning , Tahoe City, CA, Jun. 1995

    1995

  32. [40]

    Neuro-fuzzy and soft computing-a computational approach to learning and machin e intelli- gence,

    J.-S. R. Jang, C.-T. Sun, and E. Mizutani, “Neuro-fuzzy and soft computing-a computational approach to learning and machin e intelli- gence,” IEEE Trans. on Automatic Control , vol. 42, no. 10, pp. 1482– 1484, 1997

    1997

  33. [41]

    Multiple comparisons using rank sums,

    O. J. Dunn, “Multiple comparisons using rank sums,” Technometrics, vol. 6, no. 3, pp. 241–252, 1964

    1964

  34. [42]

    Controlling the false di scovery rate: A practical and powerful approach to multiple testing,

    Y . Benjamini and Y . Hochberg, “Controlling the false di scovery rate: A practical and powerful approach to multiple testing,” Journal of the Royal Statistical Society: Series B , vol. 57, no. 1, pp. 289–300, 1995

    1995

  35. [43]

    On large-batch training for deep learning: Generali zation gap and sharp minima,

    N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, a nd P . T. P . Tang, “On large-batch training for deep learning: Generali zation gap and sharp minima,” in Proc. Int’l Conf. on Learning Representations , Toulon, France, Apr. 2017

    2017

  36. [44]

    Revisiting small batch train ing for deep neural networks,

    D. Masters and C. Luschi, “Revisiting small batch train ing for deep neural networks,” arXiv preprint arXiv:1804.07612 , 2018

    2018 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.