Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves the first sample-complexity guarantees for data-driven tuning of a continuous neural-network hyperparameter, showing that the number of tasks needed grows only logarithmically in the piecewise-polynomial complexity of…

desk verdict Solid new sample complexity bounds for data-driven hyperparameter tuning in neural nets, but the activation application rests on an unverified regularity condition. read the letter →

arxiv 2501.13734 v4 pith:4CU6YJJ6 submitted 2025-01-23 cs.LG

Maria-Florina Balcan , Anh Tuan Nguyen , Dravyansh Sharma This is my paper · ORCID
classification cs.LG MSC 68Q3268T0714P10
keywords samplecomplexityhyperparametertuningdata-drivenalgorithmdesignpseudo-dimensionpiecewisepolynomialfunctionsoscillationsneuralnetworksgraph
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hyperparameter tuning in deep learning is usually guided by heuristics; this paper asks how many tasks are needed to learn a good continuous hyperparameter when each task is solved by training a network with trainable weights. The paper's central claim is that if the function $f_x(\alpha,w)$ measuring network performance on instance $x$ is piecewise polynomial in the hyperparameter $\alpha$ and the weights $w$, then the dual utility $u^*_x(\alpha)=\max_w f_x(\alpha,w)$, despite being volatile and not piecewise polynomial, has few discontinuities and local extrema. That bounded-oscillation structure converts into a pseudo-dimension bound and a PAC-style guarantee: with high probability, ERM over $m$ tasks is $O\big(\sqrt{(\log N + d\log(\Delta M) + \log(1/\delta))/m}\big)$ close to the best hyperparameter for the task distribution. The paper instantiates the framework for tuning the activation-interpolation parameter and for tuning the polynomial kernel parameter in graph convolutional networks, giving the first provable sample complexity bounds for these settings.

What carries the argument

The central object is the dual utility function $u^*_x(\alpha)=\max_{w\in W} f_x(\alpha,w)$, analysed through its oscillations: the maximum, over thresholds $z$, of the number of discontinuities of the indicator $\mathbf{1}\{u^*_x(\alpha)\ge z\}$. The machinery (i) partitions the domain of $f_x$ into monotonic curves, meaning 1-dimensional manifolds that meet each hyperplane $\alpha=\alpha_0$ at most once; (ii) uses Lagrange multiplier systems to identify candidate extrema of the piece functions along these curves; (iii) bounds the number of candidate points with Bezout and Warren-type counts on algebraic sets; and (iv) perturbs the piece functions, via Lemma 5.9, so that degeneracies can be assumed away. This turns a poorly structured max-over-weights function into a pointwise maximum over a bounded family of well-behaved curve functions, which then has local maxima only where the curve functions do, yielding the oscillation and pseudo-dimension bounds that feed the sample complexity theorem.

What would settle it

Find a concrete piecewise polynomial $f_x(\alpha,w)$ with a duplicate or tangential boundary, so Assumption 1 fails, and exhibit a dual $u^*_x(\alpha)$ with more than $O(\mathrm{poly}(\Delta,M))$ oscillations; alternatively, compute $u^*_x$ numerically for a small two-layer network with ReLU or PReLU activations, count discontinuities plus local maxima as $\alpha$ varies over $[0,1]$, and check whether the count grows faster than the theorem's polynomial rate in the degree and boundary count.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the learning-theoretic complexity of the class $\mathcal{U}=\{u_\alpha: \mathcal{X}\to[0,H] \mid \alpha\in[\alpha_{\min},\alpha_{\max}]\}$ is controlled by the geometry of the parameter-dependent dual $f_x(\alpha,w)$. Under Assumption 1, which asks that the active constraint Jacobians have full row rank and that a certain determinant polynomial is non-degenerate, the dual utility $u^*_x(\alpha)=\max_{w\in W} f_x(\alpha,w)$ is shown to have a bounded number of discontinuities and local extrema that depends only on the number of pieces $N$, the number of boundaries $M$, and their maximum degree $\Delta$. The proof decomposes the solution set of the Lagrangian system into monotonic curves, bounds the number of such curves and of their intersections using algebraic geometry, and then translates the bounds into $\mathrm{Pdim}(\mathcal{U})=O(\log N + d\log(\Delta M))$ and the stated generalization bound. The same machinery, specialized, gives Theorem 6.1 for activation-function interpolation and Theorem 6.2 for graph convolutional network kernels.

Load-bearing premise

The whole analysis rests on Assumption 1: for every selection of boundary polynomials and piece polynomial, the Jacobians of the active constraints must have full row rank at the relevant points, and a certain determinant polynomial must not vanish identically on any irreducible component; if this fails, the decomposition into monotonic curves and the counting of extrema can break down.

Editorial extensions

If this is right

  • For activation-function interpolation, tuning the continuous interpolation parameter $\alpha$ is learnable with excess loss $O\big(\sqrt{(L^2 W \log\Delta + L W\log(Tpk) + \log(1/\delta))/m}\big)$, so the number of tasks needed grows only logarithmically in network size, layer count, and number of activation breakpoints.
  • For graph convolutional networks with polynomial kernels and 0-1 loss, tuning the kernel parameter has sample complexity $O\big(\sqrt{(d_0(d+F)\log(nF\Delta)+\log(1/\delta))/m}\big)$, scaling polynomially in feature and label dimensions while only logarithmically in the number of graph vertices.
  • Both classification and regression losses fit the same framework, and the simpler piecewise-constant case yields an $O\big(\sqrt{\log(N/\delta)/m}\big)$ guarantee for any partition with $N$ connected components.
  • Because the bounds depend only on the combinatorial complexity of the piecewise polynomial structure, the results transfer to any deep-network tuning problem whose performance function has such structure, not just the two applications worked out in detail.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The conjectured removal of Assumption 1, flagged in Remark 3, would make the result fully general for piecewise polynomial duals; the perturbation construction in Lemma 5.9 already shows degeneracies can be perturbed away at arbitrarily small cost, suggesting the assumption is technical rather than intrinsic.
  • The framework assumes an ERM oracle that finds a global optimum of $f_x(\alpha,w)$ for each $\alpha$; Appendix E models flat-minima bias through a curvature-regularized surrogate that still has piecewise polynomial structure, hinting that the sample bounds may extend to realistic optimizers rather than only exact global optimization.
  • The single-hyperparameter restriction is the main bottleneck: a multidimensional analogue of bounded oscillations is left open, and without it neural architecture search over several hyperparameters has no comparable sample-complexity guarantee.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies data-driven tuning of a one-dimensional continuous hyperparameter in neural networks. For each task x the utility is defined as u_alpha(x) = max_w f_x(alpha,w), where f_x is the parameter-dependent dual function. The paper assumes f_x has piecewise-polynomial structure and develops a machinery, based on oscillations, algebraic/differential geometry, and Lagrangian constrained-optimization analysis, to bound the number of discontinuities and local maxima of the dual utility function, which yields pseudo-dimension bounds and sample complexity guarantees. The main abstract result is Theorem 5.3, which gives an O(sqrt((log N + d log(Delta M) + log(1/delta))/m)) bound under Assumption 1. The paper instantiates the framework for tuning an activation-interpolation parameter (Theorem 6.1) and for tuning a graph-polynomial-kernel parameter in GCNs (Theorem 6.2, Theorem D.3).

Significance. If Theorem 5.3 is correct, it is a substantial advance: it gives the first generic sample-complexity guarantee for data-driven tuning of a continuous hyperparameter when the utility is defined through an inner optimization over trainable weights, a setting where direct piecewise-structure analysis is much harder than in prior data-driven algorithm design. The proof machinery, including high-dimensional monotonic curves, Sard-type perturbation arguments, and a reduction from a stronger regularity assumption to Assumption 1, is nontrivial. The GCN classification result (Theorem 6.2) is self-contained and does not depend on Assumption 1, and it provides an explicit bound. However, the activation-interpolation application is conditional on an unverified regularity assumption, and the assumption can fail for standard ReLU networks at valid problem instances; as stated, the abstract's concrete claim for that application is not supported.

major comments (3)
  1. [Section 6.1 and Assumption 1] Theorem 6.1 is presented as a concrete sample-complexity bound for activation interpolation, but its hypothesis is not verified anywhere in the proof: the proof establishes only that f_x admits a piecewise-polynomial structure, not that Assumption 1 holds. The condition can in fact fail for standard ReLU networks. For a two-layer ReLU network with scalar input, weights (a,b,c,d), output c ReLU(a)+d ReLU(b), and squared-error loss on a single example with label y=0, the relevant boundaries are h1=a and h2=b. On the irreducible component Z_h={a=b=0}, the rows of J(h,nabla_w L) corresponding to partial_c L and partial_d L are identically zero because partial_c L = 2(ca+db)a and partial_d L = 2(ca+db)b vanish when a=b=0; hence the determinant in the non-degeneracy part of Assumption 1 is identically zero on a (d+1)-dimensional component of Z_h. Thus the hypothesis of Theorem 6.1 is not satisfied at the valid parameter point (a,b,c,d)=(0,0,0,0) for a standard network. The classification variant in Appendix D.1.1 has the analogous additional boundary h3=ca+db-y, and at y=0 the same point makes the relevant Jacobian rank-deficient. The paper should either prove Assumption 1 for the activation piecewise structure (possibly after excluding degenerate parameter sets) or explicitly present Theorem 6.1 and Theorem D.1 as conditional, and adjust the contribution/abstract claims accordingly.
  2. [Appendix D.2.1 and Theorem D.3] The regression GCN result is stated as an application with a concrete sample-complexity bound, but it is conditional on Assumption 2 and the appendix proves only that f_x is piecewise rational in alpha and w. No argument is given that the piecewise structure satisfies Assumption 2, which includes full-rank Jacobian conditions for the maps k and mu. Since the same regularity gap is load-bearing here as in the activation application, the theorem should be stated as conditional or the regularity conditions should be verified.
  3. [Section 5.2, Remark 3] The main theorem is conditional on Assumption 1, and Remark 3 explicitly conjectures that the assumption can be removed but provides no proof. Because neither of the two neural-network applications verifies Assumption 1, the title-level claim of sample complexity for tuning model hyperparameters in neural networks is narrower than the narrative. The introduction and abstract should state prominently that the activation-interpolation bound requires an unproven regularity condition, and that the GCN classification application is the application for which all hypotheses are verified within the paper.
minor comments (5)
  1. [Section 5.2, Assumption 1] The statement of Assumption 1 says 'piece functions fx,i, for i = 1, . . . , M' but the number of pieces is N throughout the rest of the paper; this should be corrected.
  2. [Section 3, proof of Lemma 3.1] The proof contains index typos: 'interval Ii = (alpha_i, alpha_i + 1)' should be (alpha_i, alpha_{i+1}), and the index in 'Claim 2' involving (alpha*_1, alpha*_{Ei+2}) is garbled and should be cleaned up.
  3. [Section 5.2.2, Lemma 5.9] The final bullet says 'The structure of v*_x satisfies Assumption 1', but the proof and the surrounding reduction require the constructed v*_x to satisfy Assumption 2; if the text is literal, the reduction is circular, so this should be corrected to Assumption 2.
  4. [Section 6.2 and Appendix D.2.1] The normalization of the GCN adjacency matrix is stated inconsistently: Section 6.2 uses A_hat = tilde_D^{-1} tilde_A, while Appendix D.2.1 uses tilde_D^{-1/2} tilde_A tilde_D^{-1/2}. The intended definition should be used consistently.
  5. [Section 6.2, proof of Lemma 6.3] The proof should state the condition on the distance matrix (for example, delta_{i,j} >= 0) needed to ensure that (delta_{i,j}+alpha)^Delta has strictly positive values and that the rational functions in alpha have positive denominators for alpha > 0.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the activation application's unproven Assumption 1 condition is a correctness gap, not a circular reduction.

full rationale

The derivation chain is self-contained. Theorem 5.3 is proved from the definition u*_x(alpha)=max_w f_x(alpha,w), the assumed piecewise polynomial structure, and the regularity Assumption 1; the sample-complexity bound is obtained by bounding the number of discontinuities and local maxima of the dual utility function and then applying Theorem 2.1, not by fitting any parameter or assuming the conclusion. Theorem 2.1 is a published, parameter-free transfer result from Balcan et al. 2021a; the authorship overlap is not load-bearing because the result is used as a stated tool with fixed assumptions. Similarly, Lemma B.3 is used as a standard Rademacher perturbation lemma. The activation application, Theorem 6.1, explicitly conditions on Assumption 1 ('if the piecewise structure of fx(alpha,w) satisfies Assumption 1') and does not prove that the two-layer ReLU interpolation network satisfies it; that is an unproven hypothesis and an applicability risk, not a circular step, since the theorem's conclusion is not assumed in its hypothesis. Remark 3's conjecture that Assumption 1 can be removed and Appendix E's characterization of the flatness discussion as an 'oversimplified scenario' are acknowledged limitations rather than evidence of circularity. I find no step where a fitted input is renamed a prediction, no equation that reduces to its own input by construction, and no load-bearing self-citation chain forcing the result. The central claim has independent mathematical content.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the piecewise polynomial structure of the dual function, the exact-optimizer ERM oracle, and Assumption 1. No parameters are fitted to data; the bounds depend only on structural quantities such as N, M, Delta and d, which are inputs to the problem. The regularity assumption is the most fragile part because the paper conjectures it could be removed and does not verify it for one headline application.

assumptions (6)
  • domain assumption The parameter-dependent dual function f_x(alpha,w) is piecewise polynomial in (alpha,w) with polynomial boundary functions of bounded degree (Section 5).
    Central structural premise; it excludes smooth activations (sigmoid, softmax) and non-polynomial objectives from the analysis.
  • domain assumption The dual utility is u*_x(alpha) = sup_{w in W} f_x(alpha,w) and an ERM oracle returns this exact supremum (Section 2).
    Makes u*_x deterministic; Appendix E admits this is an oversimplified scenario compared to SGD training.
  • ad hoc to paper Assumption 1: ELICQ (full row rank Jacobians for all subsets of boundary functions) and non-degeneracy (det(J(h, grad_w L)) not identically zero) hold (Section 5.2).
    Required for the manifold decomposition and Bezout counts in Theorem 5.4; Remark 3 conjectures it can be removed.
  • domain assumption There is a fixed unknown distribution D over task datasets, and all utilities take values in [0,H] (Section 2).
    Needed to phrase average-case utility and PAC-style sample complexity.
  • domain assumption The hyperparameter space is one-dimensional, alpha in [alpha_min, alpha_max] (Sections 2 and 7).
    The structural results are specifically for a single hyperparameter; multi-dimensional alpha is left as an open problem.
  • standard math Standard results from algebraic and differential geometry: Bezout's theorem, Warren's theorem, Sard's theorem, Lagrange multiplier theorem, and the preimage theorem (Appendices C.1 and C.2).
    Used unproved as background; these are established results and not contributed by the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function." pith.science (2026). https://pith.science/paper/4CU6YJJ6

@misc{pith2026250113734,
  author       = {Pith},
  title        = {Pith review of: Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CU6YJJ6}},
  note         = {Machine review of arXiv:2501.13734}
}
read the original abstract

Modern machine learning algorithms, especially deep learning based techniques, typically involve careful hyperparameter tuning to achieve the best performance. Despite the surge of intense interest in practical techniques like Bayesian optimization and random search based approaches to automating this laborious and compute intensive task, the fundamental learning theoretic complexity of tuning hyperparameters for deep neural networks is poorly understood. Inspired by this glaring gap, we initiate the formal study of hyperparameter tuning complexity in deep learning through a recently introduced data driven setting. We assume that we have a series of deep learning tasks, and we have to tune hyperparameters to do well on average over the distribution of tasks. A major difficulty is that the utility function as a function of the hyperparameter is very volatile and furthermore, it is given implicitly by an optimization problem over the model parameters. To tackle this challenge, we introduce a new technique to characterize the discontinuities and oscillations of the utility function on any fixed problem instance as we vary the hyperparameter; our analysis relies on subtle concepts including tools from differential/algebraic geometry and constrained optimization. This can be used to show that the learning theoretic complexity of the corresponding family of utility functions is bounded. We instantiate our results and provide sample complexity bounds for concrete applications tuning a hyperparameter that interpolates neural activation functions and setting the kernel parameter in graph neural networks.

Figures

Figures reproduced from arXiv: 2501.13734 by the authors.

Figure 1
Figure 1. This figure demonstrates the oscillation property for a function [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. A demonstration of the proof idea for Lemma 4.1: We begin by partitioning the domain A of the dual utility function u ∗ x(α) into intervals. This partitioning is formed using two key points for each connected component R in the partition Px of the domain A × W of fx(α, w): αR,inf = infα{α : ∃w,(α, w) ∈ R} and αR,sup = supα{α : ∃w,(α, w) ∈ R}. Given that P contains N elements, the number of such points is O(N). We de… view at source ↗
Figure 3
Figure 3. A demonstration of the proof idea for Theorem [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: A simplified illustration for the proof idea of Theorem [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generalization Guarantees for Learning Branch-and-Cut Policies in Integer Programming

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Piecewise polynomial scoring policies for branch-and-cut, including ReLU networks, yield piecewise constant cost functions with pseudo-dimension bounds that imply sample complexity guarantees.

Reference graph

Works this paper leans on

75 extracted references · 58 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Self-improving algorithms

    Nir Ailon, Bernard Chazelle, Kenneth L Clarkson, Ding Liu, Wolfgang Mulzer, and C Seshadhri. Self-improving algorithms. SIAM Journal on Computing, 40 0 (2): 0 350--375, 2011

    work page 2011

  3. [3]

    Sparse linear networks with a fixed butterfly structure: theory and practice

    Nir Ailon, Omer Leibovitch, and Vineet Nair. Sparse linear networks with a fixed butterfly structure: theory and practice. In Uncertainty in Artificial Intelligence, pages 1174--1184. PMLR, 2021

    work page 2021

  4. [4]

    Neural network learning: Theoretical foundations, volume 9

    Martin Anthony and Peter Bartlett. Neural network learning: Theoretical foundations, volume 9. cambridge University Press, 1999

    work page 1999

  5. [5]

    Designing neural network architectures using reinforcement learning

    Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. Designing neural network architectures using reinforcement learning. In International Conference on Learning Representations, 2017

    work page 2017

  6. [6]

    Data-Driven Algorithm Design

    Maria-Florina Balcan. Data-Driven Algorithm Design . In Tim Roughgarden, editor, Beyond Worst Case Analysis of Algorithms. Cambridge University Press, 2020

    work page 2020

  7. [7]

    Data driven semi-supervised learning

    Maria-Florina Balcan and Dravyansh Sharma. Data driven semi-supervised learning. Advances in Neural Information Processing Systems, 34: 0 14782--14794, 2021

    work page 2021

  8. [8]

    Learning accurate and interpretable decision trees

    Maria-Florina Balcan and Dravyansh Sharma. Learning accurate and interpretable decision trees. Uncertainty in Artificial Intelligence (UAI), 2024

    work page 2024

Show all 75 references
  1. [9]

    Sample complexity of automated mechanism design

    Maria-Florina Balcan, Tuomas Sandholm, and Ellen Vitercik. Sample complexity of automated mechanism design. Advances in Neural Information Processing Systems, 29, 2016

    2016

  2. [10]

    Learning-theoretic foundations of algorithm configuration for combinatorial partitioning problems

    Maria-Florina Balcan, Vaishnavh Nagarajan, Ellen Vitercik, and Colin White. Learning-theoretic foundations of algorithm configuration for combinatorial partitioning problems. In Conference on Learning Theory, pages 213--274. PMLR, 2017

    2017

  3. [11]

    Learning to branch

    Maria-Florina Balcan, Travis Dick, Tuomas Sandholm, and Ellen Vitercik. Learning to branch. In International Conference on Machine Learning, pages 344--353. PMLR, 2018 a

    2018

  4. [12]

    Data-driven clustering via parameterized L loyd's families

    Maria-Florina Balcan, Travis Dick, and Colin White. Data-driven clustering via parameterized L loyd's families. Advances in Neural Information Processing Systems, 31, 2018 b

    2018

  5. [13]

    A general theory of sample complexity for multi-item profit maximization

    Maria-Florina Balcan, Tuomas Sandholm, and Ellen Vitercik. A general theory of sample complexity for multi-item profit maximization. In Proceedings of the 2018 ACM Conference on Economics and Computation, pages 173--174, 2018 c

    2018

  6. [14]

    Learning to link

    Maria-Florina Balcan, Travis Dick, and Manuel Lang. Learning to link. In International Conference on Learning Representation, 2020 a

    2020

  7. [15]

    Refined bounds for algorithm configuration: The knife-edge of dual class approximability

    Maria-Florina Balcan, Tuomas Sandholm, and Ellen Vitercik. Refined bounds for algorithm configuration: The knife-edge of dual class approximability. In international Conference on Machine Learning, pages 580--590. PMLR, 2020 b

    2020

  8. [16]

    How much data is sufficient to learn high-performing algorithms? G eneralization guarantees for data-driven algorithm design

    Maria-Florina Balcan, Dan DeBlasio, Travis Dick, Carl Kingsford, Tuomas Sandholm, and Ellen Vitercik. How much data is sufficient to learn high-performing algorithms? G eneralization guarantees for data-driven algorithm design. In Proceedings of the 53rd Annual ACM SIGACT Symp...

    2021

  9. [17]

    Sample complexity of tree search configuration: Cutting planes and beyond

    Maria-Florina Balcan, Siddharth Prasad, Tuomas Sandholm, and Ellen Vitercik. Sample complexity of tree search configuration: Cutting planes and beyond. Advances in Neural Information Processing Systems, 34: 0 4015--4027, 2021 b

    2021

  10. [18]

    Provably tuning the ElasticNet across instances

    Maria-Florina Balcan, Misha Khodak, Dravyansh Sharma, and Ameet Talwalkar. Provably tuning the ElasticNet across instances. Advances in Neural Information Processing Systems, 35: 0 27769--27782, 2022 a

    2022

  11. [19]

    Structural analysis of branch-and-cut and the learnability of G omory mixed integer cuts

    Maria-Florina Balcan, Siddharth Prasad, Tuomas Sandholm, and Ellen Vitercik. Structural analysis of branch-and-cut and the learnability of G omory mixed integer cuts. Advances in Neural Information Processing Systems, 35: 0 33890--33903, 2022 b

    2022

  12. [20]

    New bounds for hyperparameter tuning of regression problems across instances

    Maria-Florina Balcan, Anh Nguyen, and Dravyansh Sharma. New bounds for hyperparameter tuning of regression problems across instances. Advances in Neural Information Processing Systems, 36, 2023

    2023

  13. [21]

    Algorithm configuration for structured P faffian settings

    Maria-Florina Balcan, Anh Tuan Nguyen, and Dravyansh Sharma. Algorithm configuration for structured P faffian settings. arXiv preprint arXiv:2409.04367, 2024

    2024 arXiv

  14. [22]

    Almost linear VC dimension bounds for piecewise polynomial networks

    Peter Bartlett, Vitaly Maiorov, and Ron Meir. Almost linear VC dimension bounds for piecewise polynomial networks. Advances in Neural Information Processing Systems, 11, 1998

    1998

  15. [23]

    Generalization bounds for data-driven numerical linear algebra

    Peter Bartlett, Piotr Indyk, and Tal Wagner. Generalization bounds for data-driven numerical linear algebra. In Conference on Learning Theory, pages 2013--2040. PMLR, 2022

    2013

  16. [24]

    Spectrally-normalized margin bounds for neural networks

    Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalized margin bounds for neural networks. Advances in Neural Information Processing Systems, 30, 2017

    2017

  17. [25]

    Nearly-tight VC -dimension and pseudodimension bounds for piecewise linear neural networks

    Peter L Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mehrabian. Nearly-tight VC -dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20 0 (63): 0 1--17, 2019

    2019

  18. [26]

    Random search for hyper-parameter optimization

    James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of machine learning research, 13 0 (2), 2012

    2012

  19. [27]

    Algorithms for hyper-parameter optimization

    James Bergstra, R \'e mi Bardenet, Yoshua Bengio, and Bal \'a zs K \'e gl. Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems, 24, 2011

    2011

  20. [28]

    Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures

    James Bergstra, Daniel Yamins, and David Cox. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In international Conference on Machine Learning, pages 115--123. PMLR, 2013

    2013

  21. [29]

    Learning from labeled and unlabeled data using graph mincuts

    Avrim Blum and Shuchi Chawla. Learning from labeled and unlabeled data using graph mincuts. In Proceedings of the Eighteenth international Conference on Machine Learning, pages 19--26, 2001

    2001

  22. [30]

    Advanced calculus

    R Creighton Buck. Advanced calculus. Waveland Press, 2003

    2003

  23. [31]

    Bert: Pre-training of deep bidirectional transformers for language understanding, 2019

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805

    2019 arXiv

  24. [32]

    Nas-bench-201: Extending the scope of reproducible neural architecture search

    Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search. In International Conference on Learning Representations, 2020

    2020

  25. [33]

    Simple and efficient architecture search for CNN s

    Thomas Elsken, Jan-Hendrik Metzen, and Frank Hutter. Simple and efficient architecture search for CNN s. In Workshop on Meta-Learning at NIPS, 2017

    2017

  26. [34]

    Neural architecture search: A survey

    Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. Journal of Machine Learning Research, 20 0 (55): 0 1--21, 2019

    2019

  27. [35]

    Neural message passing for quantum chemistry

    Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pages 1263--1272. PMLR, 2017

    2017

  28. [36]

    A PAC approach to application-specific algorithm selection

    Rishi Gupta and Tim Roughgarden. A PAC approach to application-specific algorithm selection. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, pages 123--134, 2016

    2016

  29. [37]

    Data-driven algorithm design

    Rishi Gupta and Tim Roughgarden. Data-driven algorithm design. Communications of the ACM, 63 0 (6): 0 87--94, 2020

    2020

  30. [38]

    Hyperparameter optimization: A spectral approach

    Elad Hazan, Adam Klivans, and Yang Yuan. Hyperparameter optimization: A spectral approach. ICLR, 2018

    2018

  31. [39]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026--1034, 2015

    2015

  32. [40]

    Sequential model-based optimization for general algorithm configuration

    Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In Learning and Intelligent Optimization: 5th International Conference, LION 5, Rome, Italy, January 17-21, 2011. Selected Papers 5, pages 507--523. Sp...

    2011

  33. [41]

    Learning-based low-rank approximations

    Piotr Indyk, Ali Vakilian, and Yang Yuan. Learning-based low-rank approximations. Advances in Neural Information Processing Systems, 32, 2019

    2019

  34. [42]

    Polynomial bounds for VC dimension of sigmoidal and general P faffian neural networks

    Marek Karpinski and Angus Macintyre. Polynomial bounds for VC dimension of sigmoidal and general P faffian neural networks. Journal of Computer and System Sciences, 54 0 (1): 0 169--176, 1997

    1997

  35. [43]

    Learning to relax: Setting solver parameters across a sequence of linear system instances

    Mikhail Khodak, Edmond Chow, Maria Florina Balcan, and Ameet Talwalkar. Learning to relax: Setting solver parameters across a sequence of linear system instances. In The Twelfth International Conference on Learning Representations, 2024

    2024

  36. [44]

    Fewnomials, volume 88

    Askold G Khovanski. Fewnomials, volume 88. American Mathematical Soc., 1991

    1991

  37. [45]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=SJU4ayYgl

    2017

  38. [46]

    Geometry-aware gradient algorithms for neural architecture search

    Liam Li, Mikhail Khodak, Nina Balcan, and Ameet Talwalkar. Geometry-aware gradient algorithms for neural architecture search. In International Conference on Learning Representations, 2021

    2021

  39. [47]

    Hyperband: A novel bandit-based approach to hyperparameter optimization

    Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research, 18 0 (185): 0 1--52, 2018

    2018

  40. [48]

    Learning the positions in countsketch

    Yi Li, Honghao Lin, Simin Liu, Ali Vakilian, and David Woodruff. Learning the positions in countsketch. In The Eleventh International Conference on Learning Representations, 2023

    2023

  41. [49]

    Progressive neural architecture search

    Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. In Proceedings of the European conference on computer vision (ECCV), pages 19--34, 2018

    2018

  42. [50]

    DARTS : Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS : Differentiable architecture search. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=S1eYHoC5FX

    2019

  43. [51]

    Learning algebraic multigrid using graph neural networks

    Ilay Luz, Meirav Galun, Haggai Maron, Ronen Basri, and Irad Yavneh. Learning algebraic multigrid using graph neural networks. In international Conference on Machine Learning, pages 6489--6499. PMLR, 2020

    2020

  44. [52]

    Neural nets with superlinear VC -dimension

    Wolfgang Maass. Neural nets with superlinear VC -dimension. Neural Computation, 6 0 (5): 0 877--884, 1994

    1994

  45. [53]

    NAS-Bench-Suite: NAS evaluation is (now) surprisingly easy

    Yash Mehta, Colin White, Arber Zela, Arjun Krishnakumar, Guri Zabergja, Shakiba Moradian, Mahmoud Safari, Kaicheng Yu, and Frank Hutter. NAS-Bench-Suite: NAS evaluation is (now) surprisingly easy. In International Conference on Learning Representations, 2022

    2022

  46. [54]

    Towards automatically-tuned neural networks

    Hector Mendoza, Aaron Klein, Matthias Feurer, Jost Tobias Springenberg, and Frank Hutter. Towards automatically-tuned neural networks. In Workshop on automatic machine learning, pages 58--65. PMLR, 2016

    2016

  47. [55]

    Deeparchitect: Automatically designing and training deep architectures

    Renato Negrinho and Geoff Gordon. Deeparchitect: Automatically designing and training deep architectures. arXiv preprint arXiv:1704.08792, 2017

    2017 arXiv

  48. [56]

    Efficient neural architecture search via parameters sharing

    Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In international Conference on Machine Learning, pages 4095--4104. PMLR, 2018

    2018

  49. [57]

    Convergence of stochastic processes

    David Pollard. Convergence of stochastic processes. Springer Science & Business Media, 2012

    2012

  50. [58]

    Searching for activation functions

    Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017

    2017 arXiv

  51. [59]

    Introduction to differential geometry

    Joel W Robbin and Dietmar A Salamon. Introduction to differential geometry. Springer Nature, 2022

    2022

  52. [60]

    Lagrange multipliers and optimality

    R Tyrrell Rockafellar. Lagrange multipliers and optimality. SIAM review, 35 0 (2): 0 183--238, 1993

    1993

  53. [61]

    Variational analysis, volume 317

    R Tyrrell Rockafellar and Roger J-B Wets. Variational analysis, volume 317. Springer Science & Business Media, 2009

    2009

  54. [62]

    On the density of families of sets

    Norbert Sauer. On the density of families of sets. Journal of Combinatorial Theory, Series A, 13 0 (1): 0 145--147, 1972

    1972

  55. [63]

    Understanding machine learning: From theory to algorithms

    Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge University Press, 2014

    2014

  56. [64]

    Efficiently learning the graph for semi-supervised learning

    Dravyansh Sharma and Maxwell Jones. Efficiently learning the graph for semi-supervised learning. In Uncertainty in Artificial Intelligence, pages 1900--1910. PMLR, 2023

    1900

  57. [65]

    Practical B ayesian optimization of machine learning algorithms

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical B ayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25, 2012

    2012

  58. [66]

    Scalable B ayesian optimization using deep neural networks

    Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable B ayesian optimization using deep neural networks. In international Conference on Machine Learning, pages 2171--2180. PMLR, 2015

    2015

  59. [67]

    Graph attention networks

    Petar Velic kovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li \`o , and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018

    2018

  60. [68]

    High-dimensional statistics: A non-asymptotic viewpoint, volume 48

    Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge University Press, 2019

    2019

  61. [69]

    Lower bounds for approximation by nonlinear manifolds

    Hugh E Warren. Lower bounds for approximation by nonlinear manifolds. Transactions of the American Mathematical Society, 133 0 (1): 0 167--178, 1968

    1968

  62. [70]

    Bananas: B ayesian optimization with neural architectures for neural architecture search

    Colin White, Willie Neiswanger, and Yash Savani. Bananas: B ayesian optimization with neural architectures for neural architecture search. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 10293--10301, 2021

    2021

  63. [71]

    Simplifying graph convolutional networks

    Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In international Conference on Machine Learning, pages 6861--6871. PMLR, 2019

    2019

  64. [72]

    Learning with local and global consistency

    Dengyong Zhou, Olivier Bousquet, Thomas Lal, Jason Weston, and Bernhard Sch \"o lkopf. Learning with local and global consistency. Advances in Neural Information Processing Systems, 16, 2003

    2003

  65. [73]

    Semi-supervised learning with graphs

    Xiaojin Zhu. Semi-supervised learning with graphs. Carnegie Mellon University, 2005

    2005

  66. [74]

    Semi-supervised learning using G aussian fields and harmonic functions

    Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using G aussian fields and harmonic functions. In Proceedings of the 20th international Conference on Machine Learning (ICML), pages 912--919, 2003

    2003

  67. [75]

    Neural architecture search with reinforcement learning

    Barret Zoph and Quoc Le. Neural architecture search with reinforcement learning. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=r1Ue8Hcxg

    2017

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.