Pith. sign in

REVIEW 5 major objections 6 minor 79 references

You Only Train Once

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that loss-weight hyperparameters can be trained like ordinary network parameters in a single run, and that the resulting models outperform grid-search choices on 3D depth estimation and semantic segmentation.

desk verdict A coherent but incremental recipe for gradient-based loss weighting; the generalization claims outrun the evidence, but the paper deserves a serious referee. read the letter →

arxiv 2506.04349 v1 pith:7DH7Q3IC submitted 2025-06-04 cs.LG cs.CV

Christos Sakaridis This is my paper
classification cs.LGcs.CV
keywords losshyperparameteroptimizationmulti-losstrainingsoftmaxparameterizationgradient-basedmonoculardepthestimationsemanticsegmentationcompositeone-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to remove the separate, expensive search over loss-weight hyperparameters that multi-loss models usually require. It claims that the weights in a composite training loss can be treated as ordinary trainable parameters and updated with the same gradient-based optimizer as the network, so one training run yields both the model and its loss weighting. The proposed YOTO method parameterizes the weights as a softmax of learnable log-exponents, which keeps them positive and normalized, and adds a regularization term that pulls the learned distribution toward uniform and keeps the exponents bounded. In experiments on monocular 3D depth estimation and semantic segmentation, the one-shot YOTO models outperformed the best grid-searched weight settings on held-out test data, even on evaluation metrics that are not part of the training loss. The one-shot claim is limited to loss weighting; architecture and other hyperparameters are not covered.

What carries the argument

The load-bearing object is the composite loss layer $g(\lambda, l(f(w))) = \lambda^T l(f(w))$, treated as part of the end-to-end differentiable model. Its softmax parameterization $\lambda = \mathrm{Softmax}(\mu)$ turns positive loss weights into a normalized distribution over loss terms and produces the hyperparameter gradient in Eq. (6), whose terms $l_i - l_j$ allow individual weights to increase or decrease. The accompanying regularization $L_r$ in Eq. (7) combines negated entropy, which favors a uniform distribution over losses, with softplus penalties that bound the exponents. These pieces let standard optimizers such as AdamW or SGDW update loss weights in the same loop as network weights.

What would settle it

Construct a controlled multi-task problem with known optimal loss weights and loss terms whose magnitudes differ by several orders of magnitude, then compare YOTO's converged weights with the known optimum and with a fine grid search. If the softmax gradient concentrates all weight on the largest-magnitude loss term, or if the one-shot model underperforms the best grid-search model on a held-out set, the central claim fails.

Watch

Extended reading notes

Core claim

The core discovery is that the loss weights of a composite loss $L_e = \sum_i \lambda_i l_i(f(w))$ need not be set by a separate outer loop, because the composite loss is a linear, differentiable function of those weights and can be viewed as the final layer of the network. YOTO learns $\lambda = \mathrm{Softmax}(\mu)$, treating the exponents $\mu$ as regular parameters, freezing $\mu_0 = 0$ for the basic loss, and updating the remaining exponents with the gradient in Eq. (6), which contains differences between individual loss terms and therefore can move weights up or down instead of collapsing them to zero. A regularization term combining negated entropy and softplus penalties, with a single shared hyperparameter $\rho$, keeps the softmax distribution from collapsing and the exponents bounded. In the paper's experiments, YOTO beats the best of nine grid-search weight settings for UniDepth, with margins of 0.7% in $\delta_1$ and 0.5% in $F_A$, and beats the best of thirteen grid-search settings for CISS by 0.77 mIoU on the held-out ACDC test set, while showing stable behavior across initializations and random seeds.

Load-bearing premise

The load-bearing premise is that gradients of the training composite loss with respect to the loss exponents are a reliable guide to which weight combinations generalize to unseen data, rather than a guide to which combinations merely shrink the training loss.

Editorial extensions

If this is right

  • A single training run can replace the usual grid search over loss weights for any model with a differentiable composite loss, removing the exponential cost of searching over multiple weights.
  • The learned loss weights also improve held-out metrics such as $\delta_1$, $F_A$, and mIoU that are not part of the training objective, so the method changes generalization, not just training loss.
  • YOTO layers onto existing optimizers and schedules with only one additional shared hyperparameter, the decay $\rho$, making it a minimal change to current training pipelines.
  • The converged loss weights and optimization trajectories are stable across random seeds and a wide range of initializations, so the method does not introduce a new tuning instability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The softmax gradient in Eq. (6) is driven by raw differences between loss magnitudes, so losses with very different scales could dominate the hyperparameter update regardless of usefulness; a scale-invariant variant is a natural testable extension.
  • The paper's one-shot claim covers loss-weight selection only; architecture and other non-loss hyperparameters would still need separate runs, so the practical savings depend on how much previous tuning effort went into loss weights.
  • The same mechanism could be applied to other settings where manual loss balancing is a known bottleneck, such as adversarial training or multi-task learning, but that goes beyond the paper's two vision tasks.
  • A synthetic multi-task benchmark with known optimal weights would let a reader separate the method's ability to find good weights from its ability to generalize on real vision data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes YOTO, a method for jointly optimizing loss-weight hyperparameters and network parameters in composite losses. The loss weights are reparameterized as a softmax over learned exponents, enabling standard gradient-based updates, and an entropy-plus-softplus regularizer ('hyperparameter decay') is introduced to keep the weights interior and bounded. The authors derive the relevant gradients, present an algorithm, and evaluate the method on monocular 3D/depth estimation with UniDepth and on domain-generalizing semantic segmentation with CISS, comparing against grid search. They report improvements on mean metrics for UniDepth and on a withheld test set for CISS.

Significance. The proposed parameterization is simple, the derivations in Appendices A and B are correct, and the method addresses a practical pain point in multi-loss deep learning. The segmentation evaluation is comparatively careful (three seeds and a withheld test-set submission), and the paper includes a compute statement and an initialization sensitivity analysis. However, the evidence is weaker than the claims: Table 1 contradicts the 'consistently outperforms' statement on two per-dataset metrics, the depth experiment is a single run with no variance, and the relationship between the training-loss gradient for the loss weights and test-time generalization is not analyzed. These issues need to be addressed before the paper can be recommended.

major comments (5)
  1. [Section 4.1, Table 1] The claim that YOTO 'consistently outperforms the best grid-search model across the two benchmarks and the three metrics' is contradicted by Table 1. On SUN-RGBD, the YOTO δ1 of 92.3 is below the best grid value of 92.6 (achieved by grid (1,0.01)), and the YOTO FA of 76.7 is below the best grid value of 76.9 (achieved by grid (1,0.1)). The claim should be restricted to the mean metrics or to the nuScenes benchmark, or the statement should be revised.
  2. [Section 4.1] The UniDepth comparison is based on a single training run per configuration, with no variance reported. Given that the reported margins are on the order of 0.3 to 0.7 percentage points, the reader cannot assess whether the improvement is significant. Please report multiple seeds for at least the YOTO run and the best grid-search model, or provide some measure of run-to-run variability.
  3. [Section 3.3, Eq. (6)] The HP gradient is equivalent to λ_i(l_i − L_e), which is driven by raw loss magnitudes. If the loss terms have very different scales, the gradient will systematically shift mass toward the smallest-magnitude loss regardless of its usefulness, and the paper offers no scale-normalization analysis or experiment. This is a central limitation for a method proposed as a general one-shot loss-weighting scheme. Please add a discussion of scale sensitivity and, ideally, an experiment with artificially rescaled losses.
  4. [Algorithm 1] Algorithm 1 is titled 'YOTO with SGDW with momentum' and uses a single momentum vector, but Section 4.1 states that YOTO is combined with AdamW 'analogously to Algorithm 1.' AdamW uses first and second moments, which are not present in the algorithm as written. This discrepancy makes reproduction difficult. Please provide the exact update rules used for the AdamW experiments and specify which optimizer was used for each experiment.
  5. [Section 4.1, 'significant margin'] The sentence claiming a 'significant margin of 0.7% in δ1 and 0.5% in FA' is ambiguous. Reading Table 1, those margins appear to be computed against the grid row with the original UniDepth HPs (0.25,0.1), whose FA is 59.8, rather than against the best grid FA of 60.0. Either specify the baseline or recompute the margins relative to the best grid model.
minor comments (6)
  1. [Figure 2] The sensitivity analysis in Figure 2 identifies two convergence clusters, but the text does not report the actual performance values for the five initialization seeds. Please add a small table with the metrics for each model.
  2. [Section 4.2] The hyperparameter decay ρ is set to 200 for CISS, outside the {2,20} range explored for UniDepth, with no sensitivity analysis. Please report how ρ was chosen and whether results are stable around this value.
  3. [Algorithm 1, line 12] The regularizer gradient is added after the momentum update without being accumulated in the momentum vector; the text analogizes this to decoupled weight decay, but the combination with AdamW should be made explicit for reproducibility.
  4. [Eq. (7)] The softplus term is said to ensure upper-boundedness of the optima, but a short argument or reference for why softplus (rather than an L2 penalty) is needed would help the reader.
  5. [Section 2] Given that the paper positions YOTO against gradient-based HPO methods such as Franceschi et al., an empirical comparison against at least one such method on a small problem would strengthen the claims.
  6. [Abstract] The phrase 'consistently outperforms' should be qualified to the datasets and metrics where YOTO actually improves, as per Table 1.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the softmax loss-weight gradient derivation is self-contained, and gains are evaluated on held-out benchmarks.

full rationale

The paper's derivation chain is (i) the composite loss Le = λ^T l in Eq. (1), (ii) the softmax parameterization in Eq. (5), and (iii) the HP gradients in Eq. (6), proved in Appendix A via the quotient rule. None of these steps assumes the conclusion that YOTO outperforms grid search; Eq. (6) is a direct derivative of a softmax-weighted sum and is not equivalent to any test metric by construction. The regularization gradients in Eq. (8) are likewise derived from Eq. (7) in Appendix B. The empirical claims are tested on held-out zero-shot benchmarks (nuScenes, SUN-RGBD, ACDC test) and are not fitted inputs renamed as predictions; the CISS test-set comparison even selects the worst YOTO run versus the median grid-search run. The only self-referential elements are the base methods UniDepth [58] and CISS [64] and the ACDC dataset [63], all co-authored by the present author; these citations supply experimental testbeds and baseline configurations, not the load-bearing justification for YOTO's mechanism or its generalization claim, so they do not constitute circularity. The paper also candidly acknowledges in its Discussion and Limitations that YOTO introduces two new hand-set hyperparameters (ρ and ϵ), which is a practical overhead but not a circular step. The limited statistical support in the UniDepth experiment (single seed, only four ρ/ϵ combinations tried) is a robustness concern, not circularity. Overall, the central derivation is self-contained and the comparison is external, so the appropriate finding is a low score reflecting only minor non-load-bearing self-citation.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central derivation is self-contained; the only user-chosen parameters are rho and epsilon, both introduced by YOTO. The key domain assumption is that training-loss gradients for the exponents are a useful proxy for test performance. No new physical or mathematical entities are postulated.

free parameters (2)
  • rho (hyperparameter decay) = 20 (UniDepth), 200 (CISS)
    Strength of the entropy and softplus regularizer. It is shared across all loss exponents and chosen by hand or small exploration; it directly affects the final loss weights and the reported performance.
  • epsilon (initialization for auxiliary exponents) = 0.1 (UniDepth), exp(-4) (CISS)
    Initialization scale for the auxiliary loss exponents, mu_i(0) = log(epsilon). It determines the starting point of the learned weights and is chosen by hand or small exploration; the sensitivity analysis in Figure 2 shows it can change the converged solution.
assumptions (3)
  • standard math The empirical losses l_i are differentiable with respect to model parameters w, and the softmax gradient formula (6) is correct.
    The paper proves (6) in Appendix A using the quotient rule, which is standard calculus. For neural networks with smooth losses this holds for almost all parameters.
  • domain assumption Optimizing the loss weights on the training composite loss, with the entropy regularizer, steers the model toward better generalization on test data.
    YOTO updates lambda using only training-set gradients (Eq. 6) and relies on regularization to avoid overfitting. The paper provides empirical evidence on two tasks but no guarantee; this assumption enters in Sections 3.3 and 3.4 and is evaluated in Section 4.
  • domain assumption A single shared hyperparameter decay rho sufficiently regularizes all loss exponents for diverse tasks.
    The authors choose one rho per task (rho=20 for UniDepth, rho=200 for CISS) and do not show sensitivity across a wide range. If the optimal rho varies strongly with the task or the number of losses, the claimed one-shot advantage is weakened.

how reviews work

0 comments
Cite this review

Pith. "Pith review of You Only Train Once." pith.science (2026). https://pith.science/paper/7DH7Q3IC

@misc{pith2026250604349,
  author       = {Pith},
  title        = {Pith review of: You Only Train Once},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DH7Q3IC}},
  note         = {Machine review of arXiv:2506.04349}
}
read the original abstract

The title of this paper is perhaps an overclaim. Of course, the process of creating and optimizing a learned model inevitably involves multiple training runs which potentially feature different architectural designs, input and output encodings, and losses. However, our method, You Only Train Once (YOTO), indeed contributes to limiting training to one shot for the latter aspect of losses selection and weighting. We achieve this by automatically optimizing loss weight hyperparameters of learned models in one shot via standard gradient-based optimization, treating these hyperparameters as regular parameters of the networks and learning them. To this end, we leverage the differentiability of the composite loss formulation which is widely used for optimizing multiple empirical losses simultaneously and model it as a novel layer which is parameterized with a softmax operation that satisfies the inherent positivity constraints on loss hyperparameters while avoiding degenerate empirical gradients. We complete our joint end-to-end optimization scheme by defining a novel regularization loss on the learned hyperparameters, which models a uniformity prior among the employed losses while ensuring boundedness of the identified optima. We evidence the efficacy of YOTO in jointly optimizing loss hyperparameters and regular model parameters in one shot by comparing it to the commonly used brute-force grid search across state-of-the-art networks solving two key problems in computer vision, i.e. 3D estimation and semantic segmentation, and showing that it consistently outperforms the best grid-search model on unseen test data. Code will be made publicly available.

Figures

Figures reproduced from arXiv: 2506.04349 by the authors.

Figure 1
Figure 1. Comparison of optimization dynamics of YOTO vs. grid search models for UniDepth [ [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Analysis of sensitivity of YOTO to initialization, performed for training UniDepth [ [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Comparison of YOTO vs. grid search models [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Analysis of sensitivity of YOTO to stochasticity in training, performed for training [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

79 extracted references · 70 canonical work pages

  1. [1]

    CVPR 2024 open access repository.https://openaccess.thecvf.com/CVPR2024?day=all,

    work page 2024

  2. [2]

    A new look at the statistical model identification.IEEE Transactions on Automatic Control, 19(6):716–723, 1974

    Hirotugu Akaike. A new look at the statistical model identification.IEEE Transactions on Automatic Control, 19(6):716–723, 1974. 2

    work page 1974

  3. [3]

    Max-value entropy search for multi-objective Bayesian optimization

    Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value entropy search for multi-objective Bayesian optimization. InAdvances in Neural Information Processing Systems, 2019. 3

    work page 2019

  4. [4]

    Gradient-based optimization of hyperparameters.Neural Computation, 12(8):1889–1900,

    Yoshua Bengio. Gradient-based optimization of hyperparameters.Neural Computation, 12(8):1889–1900,

    work page 1900

  5. [5]

    Algorithms for hyper-parameter optimization

    James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-parameter optimization. InAdvances in Neural Information Processing Systems, 2011. 2

    work page 2011

  6. [6]

    Small data, big decisions: Model selection in the small-data regime

    Jorg Bornschein, Francesco Visin, and Simon Osindero. Small data, big decisions: Model selection in the small-data regime. InInternational Conference on Machine Learning, 2020. 3

    work page 2020

  7. [7]

    Online learning and stochastic approximations

    Léon Bottou. Online learning and stochastic approximations. In David Saad, editor,Online Learning in Neural Networks, pages 9–42. Cambridge University Press, 1998. 1

    work page 1998

  8. [8]

    On the experimental attainment of optimum conditions.Journal of the Royal Statistical Society: Series B (Methodological), 13(1):1–38, 1951

    George EP Box and Kenneth B Wilson. On the experimental attainment of optimum conditions.Journal of the Royal Statistical Society: Series B (Methodological), 13(1):1–38, 1951. 2

    work page 1951

Show all 79 references
  1. [9]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuScenes: A multimodal dataset for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2020

  2. [10]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean Conference on Computer Vision (ECCV), 2020. 2

    2020

  3. [11]

    Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834–8...

    2018

  4. [12]

    PaLM: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1–113, 2023

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1–113, 2023. 2

    2023

  5. [13]

    The Cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benen- son, Uwe Franke, Stefan Roth, and Bernt Schiele. The Cityscapes dataset for semantic urban scene understanding. InThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR)...

    2016

  6. [14]

    Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 7

    2017

  7. [15]

    Gonzalez

    Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, and Joseph E. Gonzalez. FBNetV3: Joint architecture-recipe search using predictor pretraining. InProceedings of the IEEE/CVF Conference on Computer Vision...

    2021

  8. [16]

    Searching for a robust neural architecture in four GPU hours

    Xuanyi Dong and Yi Yang. Searching for a robust neural architecture in four GPU hours. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 10

    2019

  9. [17]

    AutoHAS: Efficient hyperparameter and architecture search

    Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, and Quoc V Le. AutoHAS: Efficient hyperparameter and architecture search. In2nd Workshop on Neural Architecture Search at International Conference on Learning Representations (ICLR), 2021. 3

    2021

  10. [18]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  11. [19]

    Depth map prediction from a single image using a multi-scale deep network

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. InAdvances in Neural Information Processing Systems, 2014. 6

    2014

  12. [20]

    Neural architecture search: A survey.Journal of Machine Learning Research, 20(55):1–21, 2019

    Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey.Journal of Machine Learning Research, 20(55):1–21, 2019. 3

    2019

  13. [21]

    Hypervolume-based expected improvement: Monotonicity properties and exact computation

    Michael TM Emmerich, André H Deutz, and Jan Willem Klinkenberg. Hypervolume-based expected improvement: Monotonicity properties and exact computation. InIEEE Congress of Evolutionary Compu- tation (CEC), 2011. 3

    2011

  14. [22]

    BOHB: Robust and efficient hyperparameter optimization at scale

    Stefan Falkner, Aaron Klein, and Frank Hutter. BOHB: Robust and efficient hyperparameter optimization at scale. InInternational Conference on Machine Learning, 2018. 2, 3

    2018

  15. [23]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022. 2

    2022

  16. [24]

    Forward and reverse gradient- based hyperparameter optimization

    Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient- based hyperparameter optimization. InInternational Conference on Machine Learning, 2017. 2

    2017

  17. [25]

    Bilevel programming for hyperparameter optimization and meta-learning

    Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. InInternational Conference on Machine Learning, 2018

    2018

  18. [26]

    Hyperparameter optimization in machine learning.CoRR, abs/2410.22854, 2024

    Luca Franceschi, Michele Donini, Valerio Perrone, Aaron Klein, Cédric Archambeau, Matthias Seeger, Massimiliano Pontil, and Paolo Frasconi. Hyperparameter optimization in machine learning.CoRR, abs/2410.22854, 2024. 2

    2024

  19. [27]

    Bayesian optimization with inequality constraints

    Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunningham. Bayesian optimization with inequality constraints. InInternational Conference on Machine Learning,

  20. [28]

    Bayesian optimization with unknown constraints

    Michael A Gelbart, Jasper Snoek, and Ryan P Adams. Bayesian optimization with unknown constraints. InConference on Uncertainty in Artificial Intelligence (UAI), 2014. 3

    2014

  21. [29]

    Fast R-CNN

    Ross Girshick. Fast R-CNN. InInternational Conference on Computer Vision (ICCV), 2015. 2

    2015

  22. [30]

    On the iteration complexity of hypergradient computation

    Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil, and Saverio Salzo. On the iteration complexity of hypergradient computation. InInternational Conference on Machine Learning, 2020. 2

    2020

  23. [31]

    Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation

    Nikolaus Hansen and Andreas Ostermeier. Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. InProceedings of IEEE International Conference on Evolutionary Computation, 1996. 2

    1996

  24. [32]

    Mask R-CNN

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. InProceedings of the IEEE International Conference on Computer Vision (ICCV), 2017. 2

    2017

  25. [33]

    Predictive entropy search for multi-objective Bayesian optimization

    Daniel Hernández-Lobato, Jose Hernández-Lobato, Amar Shah, and Ryan Adams. Predictive entropy search for multi-objective Bayesian optimization. InInternational Conference on Machine Learning, 2016. 3

    2016

  26. [34]

    Sequential model-based optimization for general algorithm configuration

    Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. InInternational Conference on Learning and Intelligent Optimization, 2011. 2

    2011

  27. [35]

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 2

    2017

  28. [36]

    Bag of baselines for multi-objective joint neural architecture search and hyperparameter optimization

    Sergio Izquierdo, Julia Guerrero-Viu, Sven Hauns, Guilherme Miotto, Simon Schrodi, André Biedenkapp, Thomas Elsken, Difan Deng, Marius Lindauer, and Frank Hutter. Bag of baselines for multi-objective joint neural architecture search and hyperparameter optimization. In8th ICML ...

    2021

  29. [37]

    Population based training of neural networks.CoRR, abs/1711.09846, 2017

    Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, and Koray Kavukcuoglu. Population based training of neural networks.CoRR, abs/1711.09846, 2017. 2

    2017 arXiv

  30. [38]

    Non-stochastic best arm identification and hyperparameter opti- mization

    Kevin Jamieson and Ameet Talwalkar. Non-stochastic best arm identification and hyperparameter opti- mization. InArtificial Intelligence and Statistics, 2016. 3

    2016

  31. [39]

    Almost optimal exploration in multi-armed bandits

    Zohar Karnin, Tomer Koren, and Oren Somekh. Almost optimal exploration in multi-armed bandits. In International Conference on Machine Learning, 2013. 3

    2013

  32. [40]

    Statistical improvement criteria for use in multiobjective design optimization.AIAA, 44(4): 879–891, 2006

    Andy J Keane. Statistical improvement criteria for use in multiobjective design optimization.AIAA, 44(4): 879–891, 2006. 3

    2006

  33. [41]

    3D Gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 42(4):1–14, 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 42(4):1–14, 2023. 2

    2023

  34. [42]

    Fast Bayesian optimization of machine learning hyperparameters on large datasets

    Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, and Frank Hutter. Fast Bayesian optimization of machine learning hyperparameters on large datasets. InArtificial Intelligence and Statistics, 2017. 2, 3

    2017

  35. [43]

    ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems.IEEE Transactions on Evolutionary Computation, 10(1):50–66,

    Joshua Knowles. ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems.IEEE Transactions on Evolutionary Computation, 10(1):50–66,

  36. [44]

    Adaptive regularization in neural network modeling

    Jan Larsen, Claus Svarer, Lars Nonboe Andersen, and Lars Kai Hansen. Adaptive regularization in neural network modeling. InNeural Networks: Tricks of the Trade, pages 113–132. Springer, 1998. 2

    1998

  37. [45]

    Orr, and Klaus-Robert Müller

    Yann LeCun, Léon Bottou, Geneviève B. Orr, and Klaus-Robert Müller. Efficient backprop. In Geneviève B. Orr and Klaus-Robert Müller, editors,Neural Networks: Tricks of the Trade, volume 1524 ofLecture Notes in Computer Science, pages 9–50. Springer, 1998. 1

    1998

  38. [46]

    Constrained Bayesian optimization with noisy experiments.Bayesian Analysis, 14(2):495–519, 2019

    Benjamin Letham, Brian Karrer, Guilherme Ottoni, and Eytan Bakshy. Constrained Bayesian optimization with noisy experiments.Bayesian Analysis, 14(2):495–519, 2019. 3

    2019

  39. [47]

    Hyperband: A novel bandit-based approach to hyperparameter optimization.Journal of Machine Learning Research, 18 (185):1–52, 2018

    Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization.Journal of Machine Learning Research, 18 (185):1–52, 2018. 3

    2018

  40. [48]

    A system for massively parallel hyperparameter tuning.CoRR, abs/1810.05934, 2018

    Lisha Li, Kevin Jamieson, Afshin Rostamizadeh, Katya Gonina, Moritz Hardt, Benjamin Recht, and Ameet Talwalkar. A system for massively parallel hyperparameter tuning.CoRR, abs/1810.05934, 2018. 3

    2018 arXiv

  41. [49]

    Reviving and improving recurrent back-propagation

    Renjie Liao, Yuwen Xiong, Ethan Fetaya, Lisa Zhang, KiJung Yoon, Xaq Pitkow, Raquel Urtasun, and Richard Zemel. Reviving and improving recurrent back-propagation. InInternational Conference on Machine Learning, 2018. 2

    2018

  42. [50]

    DARTS: Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. In International Conference on Learning Representations, 2019. 1, 3

    2019

  43. [51]

    Fully convolutional networks for semantic segmenta- tion

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmenta- tion. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. 8, 9

    2015

  44. [52]

    Particle swarm optimization for hyper-parameter selection in deep neural networks

    Pablo Ribalta Lorenzo, Jakub Nalepa, Michal Kawulok, Luciano Sanchez Ramos, and José Ranilla Pastor. Particle swarm optimization for hyper-parameter selection in deep neural networks. InProceedings of the Genetic and Evolutionary Computation Conference, 2017. 2

    2017

  45. [53]

    CMA-ES for hyperparameter optimization of deep neural networks

    Ilya Loshchilov and Frank Hutter. CMA-ES for hyperparameter optimization of deep neural networks. CoRR, abs/1604.07269, 2016. 2

    2016 arXiv

  46. [54]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019. 2, 5, 6, 8

    2019

  47. [55]

    Gradient-based hyperparameter optimization through reversible learning

    Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-based hyperparameter optimization through reversible learning. InInternational Conference on Machine Learning, 2015. 1, 2

    2015

  48. [56]

    Hyperparameter optimization with approximate gradient

    Fabian Pedregosa. Hyperparameter optimization with approximate gradient. InInternational Conference on Machine Learning, 2016. 1, 2

    2016

  49. [57]

    Constrained Bayesian optimization with max-value entropy search.arXiv preprint arXiv:1910.07003,

    Valerio Perrone, Iaroslav Shcherbatyi, Rodolphe Jenatton, Cedric Archambeau, and Matthias Seeger. Constrained Bayesian optimization with max-value entropy search.arXiv preprint arXiv:1910.07003,

    1910 arXiv

  50. [58]

    UniDepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 6, 7, 8

    2024

  51. [59]

    Meta-learning with implicit gradients

    Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. InAdvances in Neural Information Processing Systems, 2019. 1

    2019

  52. [60]

    A comprehensive survey of neural architecture search: Challenges and solutions.ACM Comput

    Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-yao Huang, Zhihui Li, Xiaojiang Chen, and Xin Wang. A comprehensive survey of neural architecture search: Challenges and solutions.ACM Comput. Surv., 54(4), May 2021. 3

    2021

  53. [61]

    Faster R-CNN: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. InAdvances in Neural Information Processing Systems, 2015. 2

    2015

  54. [62]

    Rumelhart, Geoffrey E

    David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back- propagating errors.Nature, 323(6088):533–536, 1986. 1

    1986

  55. [63]

    ACDC: The Adverse Conditions Dataset with Correspondences for semantic driving scene understanding

    Christos Sakaridis, Dengxin Dai, and Luc Van Gool. ACDC: The Adverse Conditions Dataset with Correspondences for semantic driving scene understanding. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 9

    2021

  56. [64]

    Condition-invariant semantic segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):3111–3125, 2025

    Christos Sakaridis, David Bruggemann, Fisher Yu, and Luc Van Gool. Condition-invariant semantic segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):3111–3125, 2025. 8, 9, 16, 17

    2025

  57. [65]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.CoRR, abs/1910.01108, 2019

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.CoRR, abs/1910.01108, 2019. 2

    1910 arXiv

  58. [66]

    Practical Bayesian optimization of machine learning algorithms

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical Bayesian optimization of machine learning algorithms. InAdvances in Neural Information Processing Systems, 2012. 2

    2012

  59. [67]

    Lichtenberg, and Jianxiong Xiao

    Shuran Song, Samuel P. Lichtenberg, and Jianxiong Xiao. SUN RGB-D: A RGB-D scene understanding benchmark suite. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015. 6, 7

    2015

  60. [68]

    Scalability in perception for autonomous driving: Waymo Open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang,...

  61. [69]

    Kevin Swersky, Jasper Snoek, and Ryan P. Adams. Multi-task Bayesian optimization. InAdvances in Neural Information Processing Systems, 2013. 3

    2013

  62. [70]

    Freeze-thaw Bayesian optimization.CoRR, abs/1406.3896, 2014

    Kevin Swersky, Jasper Snoek, and Ryan Prescott Adams. Freeze-thaw Bayesian optimization.CoRR, abs/1406.3896, 2014. 3

    2014 arXiv

  63. [71]

    Learning to mutate with hypergradient guided population

    Zhiqiang Tao, Yaliang Li, Bolin Ding, Ce Zhang, Jingren Zhou, and Yun Fu. Learning to mutate with hypergradient guided population. InAdvances in Neural Information Processing Systems, 2020. 2

    2020

  64. [72]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Argoverse 2: Next generation datasets for self-driving perception and fo...

    2021

  65. [73]

    Esperança, and Fabio M

    Antoine Yang, Pedro M. Esperança, and Fabio M. Carlucci. NAS evaluation is frustratingly hard. In International Conference on Learning Representations, 2020. 3

    2020

  66. [74]

    Towards automated deep learning: Efficient joint neural architecture and hyperparameter search.CoRR, abs/1807.06906, 2018

    Arber Zela, Aaron Klein, Stefan Falkner, and Frank Hutter. Towards automated deep learning: Efficient joint neural architecture and hyperparameter search.CoRR, abs/1807.06906, 2018. 3

    2018 arXiv

  67. [75]

    Un- derstanding and robustifying differentiable architecture search

    Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter. Un- derstanding and robustifying differentiable architecture search. InInternational Conference on Learning Representations, 2020. 3

    2020

  68. [76]

    Random hypervolume scalarizations for provable multi-objective black box optimization

    Richard Zhang and Daniel Golovin. Random hypervolume scalarizations for provable multi-objective black box optimization. InInternational Conference on Machine Learning, 2020. 3 13

    2020

  69. [77]

    EcoNAS: Finding proxies for economical neural architecture search

    Dongzhan Zhou, Xinchi Zhou, Wenwei Zhang, Chen Change Loy, Shuai Yi, Xuesen Zhang, and Wanli Ouyang. EcoNAS: Finding proxies for economical neural architecture search. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 3

    2020

  70. [78]

    transition

    Barret Zoph and Quoc V . Le. Neural architecture search with reinforcement learning. InInternational Conference on Learning Representations, 2017. 1, 3 14 A Derivation of Composite Empirical Loss Gradients We derive the expression which is provided in (6) for the gradients of ...

    2017

  71. [2024]

    Accessed: 2025-05-15. 2

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.