REVIEW 5 major objections 6 minor 48 references
CAdam: Confidence-Based Optimization for Online Learning
T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read CAdam replaces Adam's update with a per-coordinate confidence gate that pauses updates when momentum and gradient disagree, and the paper argues this makes online learning adapt faster to distribution shifts and more resilient to noise.
desk verdict CAdam is a clean empirical paper about a one-line Adam variant whose deterministic-only theory is narrower than the abstract suggests, but the authors disclose the gap and the production evidence is real. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the confidence mask $M_t = I(m_t \odot g_t > 0)$, applied element-wise to the bias-corrected momentum before the update: $$\theta_t = \theta_{t-1} - \$\alpha$ \frac{\hat m_t \odot M_t}{\sqrt{\hat v_t} + \epsilon}.$$ The mask is hyperparameter-free: it either keeps Adam's original update or blocks a coordinate for one step. It does the argument's work in two places. Practically, it creates the pause-and-monitor behavior that the paper identifies with distinguishing distribution shift from noise. Theoretically, it only shrinks the update vector, so the displacement bound $\|x_{t+1}-x_t\| \le \alpha D$ still holds, which is why the proof can reuse Adam's convergence framework and only needs the two masked-update lemmas.
What would settle it
Run CAdam versus Adam on a stationary smooth nonconvex problem with stochastic gradients having zero mean (for example, a quadratic with symmetric label noise), and track the average squared gradient norm $\frac{1}{T}\sum_{t=1}^T \|\nabla f(x_t)\|^2$. If CAdam's average stays above Adam's at large $T$, or if its updates stall on coordinates where momentum and gradient differ only because of noise, then the claimed same convergence rate does not transfer to the stochastic setting.
Extended reading notes
Core claim
CAdam's central claim is that the alignment between momentum and current gradient is a reliable confidence signal for online optimization. On coordinates where $m_t$ and $g_t$ agree, CAdam performs the exact Adam update; on coordinates where they disagree, it withholds the update for that step. The paper argues that this selective updating prevents stale momentum from carrying the model in an outdated direction after a distribution shift, and prevents a single corrupted gradient from moving the parameters, while repeated disagreements let the momentum decay and re-align with the new gradient trend. Under Assumptions 4.1 and 4.2, Theorem 4.3 gives an average squared-gradient bound of order $\gamma^2$ after $T=\max\{(1-\beta_1)^{-2}, O(\alpha \gamma^{-2})\}$ iterations, matching the convergence rate of Adam in the deterministic setting; the proof supplies two lemmas bounding the masked update and substitutes them into the relaxed-smoothness framework. The empirical sections report consistent gains, including a $0.30\%$ average GAUC improvement over Adam across seven production A/B tests.
Load-bearing premise
The proof that CAdam keeps Adam's convergence rate assumes deterministic gradients, while the paper's target use is stochastic online learning; if the guarantee does not extend to stochastic gradients, the theoretical support for CAdam in the setting where it is claimed to help is absent.
Editorial extensions
If this is right
- CAdam can be swapped into a live system in place of Adam or AdamW with no hyperparameter changes, because the mask is the only added mechanism and it defaults to Adam on aligned coordinates.
- Under sudden and continuous distribution shifts, the alignment ratio drops at the moment of the shift and recovers afterward; CAdam's accuracy drops more slowly and recovers faster than Adam's.
- Under label noise, CAdam updates fewer parameters as noise increases, and on the noisy Criteo benchmark its average AUC drop is $-0.12$ compared with Adam's $-0.39$.
- The convergence guarantee is limited to deterministic objectives: Theorem 4.3 establishes the same rate as Adam but does not claim a strictly better rate, and the paper leaves stochastic convergence analysis to future work.
- In production recommendation scenarios, the average GAUC gain over Adam was $0.30\%$, and CAdam has been running in 16 online scenarios for over nine months.
Reading between the lines
- If the deterministic proof does not transfer to stochastic gradients, the mask may still confer stability, but one would expect CAdam to be conservative on stationary stochastic problems: with symmetric zero-mean noise, roughly half the coordinates will have $m_t \cdot g_t \le 0$ at any step, effectively halving the update magnitude on those coordinates.
- The sign-agreement gate is a generic component, so it likely transfers to other momentum-based optimizers such as Lion, SGDM, and RAdam; the paper's own ablations with CSGDM and CAmsGrad are initial evidence.
- A testable extension is to tune the gate to a momentum-yield threshold rather than a hard sign: for high-noise regimes, requiring agreement over several consecutive steps could filter persistent noise, while for fast drift, a single-step mask may be too conservative.
- In recommendation production, part of the observed GMV and GAUC gain may come from the optimizer dampening updates after abrupt user-behavior changes; a deployment experiment could measure whether CAdam's advantage grows with measured concept-drift rate or label-corruption rate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CAdam, a confidence-based variant of Adam that masks coordinate-wise updates whenever the running momentum and the current gradient disagree in sign (Algorithm 1, line 14). The authors claim that CAdam preserves Adam's convergence rate under relaxed nonconvex smoothness assumptions (Theorem 4.3), adapts faster to distribution shifts, is more robust to label noise, and outperforms Adam and other optimizers in offline benchmarks and in a seven-scenario production A/B test. The convergence proof is developed in Appendix B by adapting the framework of Li et al. [24] to the masked update, and the authors explicitly acknowledge in the Limitation section that the analysis is currently restricted to a deterministic setting.
Significance. If the empirical and theoretical claims were fully established, CAdam would be a practically attractive drop-in replacement for Adam: it introduces no new hyperparameters, the update rule is simple, and the reported nine-month production deployment across sixteen scenarios is a strong real-world endorsement. The paper also deserves credit for being explicit about its limitation that the convergence analysis is deterministic and that no formal superiority guarantee over Adam is proved. However, the central theoretical result does not cover the stochastic, non-stationary online setting that motivates the paper, and the reported offline gains on Criteo are small relative to the reported standard deviations. The production A/B results are valuable but are presented without statistical detail, and one table contains an arithmetic inconsistency. The paper would need a substantive revision to make the theoretical and empirical support match the strength of the claims.
major comments (5)
- [§4, Theorem 4.3 and Appendix A] The paper's target setting is online learning with distribution shifts and noisy labels, as stated in the abstract and Section 1, but Theorem 4.3 is a convergence bound for a single fixed nonconvex objective f under the deterministic smoothness condition of Assumption 4.2. The Limitation section explicitly says that the analysis is established under a deterministic setting. A bound on the average squared gradient of a static f does not imply any regret or tracking guarantee for a shifting sequence of objectives f_t, nor does it control the effect of stochastic gradient noise. As written, the theoretical support for CAdam in its intended deployment scenario is absent. The authors should either provide a stochastic or dynamic analysis, or explicitly scope the theoretical claims to offline deterministic optimization and soften the online-learning conclusions accordingly.
- [Appendix B, Eq. (4), Eq. (5), Lemma B.3] The proof that the mask does not break convergence relies on the deterministic inequality ||π_{t,Ξ_t}|| ≤ ||π_t|| in Lemma B.3. Under stochastic gradients this inequality is not sufficient: the event {m_{t,i} g_{t,i} > 0} is correlated with g_{t,i}, so the masked update is a truncated, biased function of the gradient, and the variance of the masked noise term can exceed the Adam error ||π_t||^2 in low-signal coordinates. The proof supplies no stochastic descent inequality, and the probability notation in Theorem 4.3 is not connected to any source of randomness in the deterministic proof. A concrete fix would be to prove a descent lemma under unbiased stochastic gradients or to state the theorem as a deterministic result without the δ-probability language.
- [Algorithm 1 vs. Appendix B, Eq. (4)] Algorithm 1 line 15 uses the AMSGrad-style denominator sqrt(v_hat_max,t) + epsilon, while the proof in Eq. (4) analyzes the update with sqrt(v_hat_t) + epsilon and never mentions v_hat_max. Consequently, the proof does not cover the AMSGrad variant CAmsGrad that is reported in Table 1 and Table 4, even though the paper presents CAmsGrad as a main empirical contribution. The proof should either explicitly handle the v_hat_max recursion or the theorem should be restricted to the non-AMSGrad version of CAdam.
- [§3.3, Table 1] The headline empirical claim that CAdam and CAmsGrad 'surpass' Adam is not supported by the reported statistics. In Table 1, Adam has average AUC 80.92 and CAdam 80.94, but for several models the differences are smaller than the reported standard deviations (e.g., DeepFM: 80.87±.011 vs 80.88±.008; DNN: 80.89±.003 vs 80.90±.002). No significance tests, paired comparisons, or confidence intervals are reported, and the authors state only that results are averaged over three seeds. Given that the paper claims a 0.05% GAUC increase is nontrivial, the statistical evidence for the Criteo result needs to be made explicit rather than asserted.
- [§3.4, Table 3] The production A/B table has an arithmetic inconsistency that needs correction or explanation. For the seven listed scenarios, the arithmetic mean of the Adam GAUC values is 85.91%, not the reported 85.34%, and the mean of the CAdam values is 86.17%, not the reported 85.64%; the reported average improvement of 0.30% is also larger than the 0.26% implied by the per-scenario differences. If the 'Average' column is a traffic-weighted or otherwise non-arithmetic mean, the weighting should be described; otherwise the production results are not reproducible from the table as printed.
minor comments (6)
- [§2, Algorithm 1] The notation in the algorithm mixes masked and unmasked quantities: line 14 overwrites hat_m_t with the masked vector, but the text in Section 2 describes the update as using m_t/sqrt(v_t), which is not exactly what the algorithm computes. Please unify the notation between the algorithm, the descriptive text, and the proof.
- [§3.1 and Appendix D.1] The numerical noise experiment is described differently in Section 3.1 (randomly modifying the function value) and in Appendix D.1 (randomly masking each gradient dimension with probability 0.5). Please clarify which procedure was actually used, since the two descriptions lead to different noise models.
- [§3.1 and Figure 2] The text mentions that CAdam exhibits 'lower regret' under distribution shifts, but no regret measure is defined or plotted in Figure 2. Adding a quantitative regret or tracking-error curve would strengthen the numerical illustration.
- [References [12]] Reference [12] has a malformed author list ('Jean-Baptiste Tien, joycenv') and should be corrected to the proper Criteo Display Advertising Challenge citation.
- [§3.3, D.4] The hyperparameter selection protocol is described at different levels of detail: Section D.3 states that Adam's optimal learning rate was used for both Adam and CAdam, while Section D.4 describes a per-optimizer grid search. Please state explicitly for each experiment whether the learning rate was tuned separately for CAdam, since this affects the fairness interpretation of the comparisons.
- [§3.4] The paper reports that CAdam has been deployed in 16 scenarios but provides A/B results only for 7 scenarios; please indicate whether the remaining 9 deployments were evaluated similarly and whether the reported nine-month stability claim refers to all 16 scenarios.
Circularity Check
No significant circularity: CAdam's mask is defined independently of its convergence theorem, and the proof leans on an external framework without self-citation chains or fitted parameters.
full rationale
CAdam's defining operation is the element-wise gate I(m_t ⊙ g_t > 0) in Algorithm 1, line 14, which is a fixed deterministic rule depending only on the momentum and gradient at the current step. Nothing in this gate is fitted to the performance results or to the convergence bound: Theorem 4.3 is proved for a fixed nonconvex f under Assumptions 4.1 and 4.2, following the external deterministic framework of Li et al. [24], and the two supplied lemmas (B.2 and B.3) only show that the masked update is no larger in norm than Adam's update. The key inequality ‖π_{t,Ξ_t}‖ ≤ ‖π_t‖ is a coordinate-restriction fact, not an assumption of the desired conclusion. There are no fitted inputs renamed as predictions, and the citations to [24] and [25] are to non-overlapping external authors, so the self-citation patterns do not apply. The Limitation section's admission that the convergence analysis is 'currently established under a deterministic setting' narrows the theorem's scope to offline deterministic optimization and leaves the stochastic online guarantee unsupported, but a scope limitation is not circularity: the claimed result follows from its stated assumptions without assuming itself. The interpretive story about distinguishing distribution shifts from noise is a post-hoc explanation of the gate and is not used as an input to the derivation. Accordingly, the paper is self-contained in its derivation chain and receives a score of 0.
Assumptions & free parameters
assumptions (4)
- domain assumption Assumption 4.1: f is differentiable, closed, and bounded from below (Delta_1 = f(x_1) - f* < infinity).
- domain assumption Assumption 4.2: f is (rho, L0, Lrho)-smooth with 0 <= rho < 2, i.e. ||grad^2 f(x)|| <= L0 + Lrho ||grad f(x)||^rho.
- standard math Lemma C.2 from Li et al. [24]: the bias-corrected momentum estimate satisfies ||mhat_t|| <= G under the stated conditions.
- domain assumption The convergence analysis is restricted to the deterministic setting; stochastic gradients are not analyzed.
Cite this review
Pith. "Pith review of CAdam: Confidence-Based Optimization for Online Learning." pith.science (2026). https://pith.science/paper/XGM42TIW
@misc{pith2026241119647,
author = {Pith},
title = {Pith review of: CAdam: Confidence-Based Optimization for Online Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/XGM42TIW}},
note = {Machine review of arXiv:2411.19647}
}
abstract
Modern recommendation systems frequently employ online learning to dynamically update their models with freshly collected data. The most commonly used optimizer for updating neural networks in these contexts is the Adam optimizer, which integrates momentum ($m_t$) and adaptive learning rate ($v_t$). However, the volatile nature of online learning data, characterized by its frequent distribution shifts and presence of noise, poses significant challenges to Adam's standard optimization process: (1) Adam may use outdated momentum and the average of squared gradients, resulting in slower adaptation to distribution changes, and (2) Adam's performance is adversely affected by data noise. To mitigate these issues, we introduce CAdam, a confidence-based optimization strategy that assesses the consistency between the momentum and the gradient for each parameter dimension before deciding on updates. If momentum and gradient are in sync, CAdam proceeds with parameter updates according to Adam's original formulation; if not, it temporarily withholds updates and monitors potential shifts in data distribution in subsequent iterations. This method allows CAdam to distinguish between the true distributional shifts and mere noise, and to adapt more quickly to new data distributions. In various settings with distribution shift or noise, our experiments demonstrate that CAdam surpasses other well-known optimizers, including the original Adam. Furthermore, in large-scale A/B testing within a live recommendation system, CAdam significantly enhances model performance compared to Adam, leading to substantial increases in the system's gross merchandise volume (GMV).
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[24]
Convergence of adam under relaxed assumptions
Haochuan Li, Alexander Rakhlin, and Ali Jadbabaie. Convergence of adam under relaxed assumptions. Advances in Neural Information Processing Systems, 36:52166–52196, 2023
2023
-
[1]
A survey of recommendation systems: recommendation models, techniques, and application fields
Hyeyoung Ko, Suyeon Lee, Yoonseo Park, and Anna Choi. A survey of recommendation systems: recommendation models, techniques, and application fields. Electronics, 11(1):141, 2022
work page 2022
-
[2]
Kingma and Jimmy Ba
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations (ICLR), 2015
2015
-
[3]
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy Alexey. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2020
arXiv 2010
-
[4]
Attention is all you need
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017
2017
-
[5]
Proximal policy optimization algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017
arXiv 2017
-
[6]
Learning Robust Recommender from Noisy Implicit Feedback
Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. Learning robust recommender from noisy implicit feedback. arXiv preprint arXiv:2112.01160, 2021
work page Pith review arXiv 2021
-
[7]
A Gradient-based Approach for Online Robust Deep Neural Network Training with Noisy Labels
Yifan Yang, Alec Koppel, and Zheng Zhang. A gradient-based approach for online robust deep neural network training with noisy labels. arXiv preprint arXiv:2306.05046, 2023
work page Pith review arXiv 2023
Show all 48 references
-
[8]
On the convergence of adam and beyond
Sashank J Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of adam and beyond. In International Conference on Learning Representations, 2018
2018
-
[9]
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014
2014 arXiv
-
[10]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
-
[11]
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017
2017
-
[12]
Display advertising challenge, 2014
Olivier Chapelle Jean-Baptiste Tien, joycenv. Display advertising challenge, 2014. URL https://kaggle.com/competitions/criteo-display-ad-challenge
2014
-
[13]
Open benchmarking for click-through rate prediction
Jieming Zhu, Jinyang Liu, Shuai Yang, Qi Zhang, and Xiuqiang He. Open benchmarking for click-through rate prediction. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 2759–2769, 2021
2021
-
[14]
Online robust and adaptive learning from data streams
Shintaro Fukushima, Atsushi Nitanda, and Kenji Yamanishi. Online robust and adaptive learning from data streams. arXiv preprint arXiv:2007.12160, 2020. 10
2007 arXiv
-
[15]
Adaptive methods for nonconvex optimization
Manzil Zaheer, Sashank Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar. Adaptive methods for nonconvex optimization. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Sys- tems, volume...
2018
-
[16]
On the variance of the adaptive learning rate and beyond
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265, 2019
1908 arXiv
-
[17]
Adabelief optimizer: Adapting stepsizes by the belief in observed gradients
Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar C Tatikonda, Nicha Dvornek, Xenophon Papademetris, and James Duncan. Adabelief optimizer: Adapting stepsizes by the belief in observed gradients. Advances in neural information processing systems , 33:18795–18806, 2020
2020
-
[18]
Deepfm: a factorization-machine based neural network for ctr prediction
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247, 2017
2017 arXiv
-
[19]
Wide & deep learning for recommender systems
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, pag...
2016
-
[20]
Deep neural networks for youtube recommenda- tions
Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommenda- tions. In Proceedings of the 10th ACM conference on recommender systems, pages 191–198, 2016
2016
-
[21]
Product- based neural networks for user response prediction
Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. Product- based neural networks for user response prediction. In 2016 IEEE 16th international conference on data mining (ICDM), pages 1149–1154. IEEE, 2016
2016
-
[22]
Deep & cross network for ad click predictions
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In Proceedings of the ADKDD’17, pages 1–7. 2017
2017
-
[23]
Symbolic discovery of optimization algorithms
Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, et al. Symbolic discovery of optimization algorithms. Advances in neural information processing systems, 36:49205–49233, 2023
2023
-
[25]
On the convergence of adam under non-uniform smoothness: Separability from sgdm and beyond
Bohan Wang, Huishuai Zhang, Qi Meng, Ruoyu Sun, Zhi-Ming Ma, and Wei Chen. On the convergence of adam under non-uniform smoothness: Separability from sgdm and beyond. arXiv preprint arXiv:2403.15146, 2024
2024 arXiv
-
[26]
Decoupled weight decay regularization
I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017
2017 arXiv
-
[27]
Adaptive methods for nonconvex optimization
Manzil Zaheer, Sashank Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar. Adaptive methods for nonconvex optimization. Advances in neural information processing systems, 31, 2018
2018
-
[28]
Cautious optimizers: Improving training with one line of code
Kaizhao Liang, Lizhang Chen, Bo Liu, and Qiang Liu. Cautious optimizers: Improving training with one line of code. arXiv preprint arXiv:2411.16085, 2024
2024
-
[29]
Learning under concept drift: A review
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE transactions on knowledge and data engineering , 31(12): 2346–2363, 2018
2018
-
[30]
Learning from time-changing data with adaptive windowing
Albert Bifet and Ricard Gavalda. Learning from time-changing data with adaptive windowing. In Proceedings of the 2007 SIAM international conference on data mining , pages 443–448. SIAM, 2007. 11
2007
-
[31]
A streaming ensemble algorithm (sea) for large-scale classification
W Nick Street and YongSeog Kim. A streaming ensemble algorithm (sea) for large-scale classification. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 377–382, 2001
2001
-
[32]
Online convex programming and generalized infinitesimal gradient ascent
Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03), pages 928–936, 2003
2003
-
[33]
Model-agnostic meta-learning for fast adap- tation of deep networks
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In International conference on machine learning, pages 1126–1135. PMLR, 2017
2017
-
[34]
A case study of batch and incremental recom- mender systems in supermarket data under concept drifts and cold start
Antˆonio David Viniski, Jean Paul Barddal, Alceu de Souza Britto Jr, Fabr ´ıcio Enembreck, and Humberto Vinicius Aparecido de Campos. A case study of batch and incremental recom- mender systems in supermarket data under concept drifts and cold start. Expert Systems with Applic...
2021
-
[35]
Robust loss functions under label noise for deep neural networks
Aritra Ghosh, Himanshu Kumar, and P Shanti Sastry. Robust loss functions under label noise for deep neural networks. In Proceedings of the AAAI conference on artificial intelligence , volume 31, 2017
2017
-
[36]
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958, 2014
1929
-
[37]
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Michael Gutmann and Aapo Hyv ¨arinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international con- ference on artificial intelligence and statistics, pages 297–304. JMLR Workshop and Conferenc...
2010
-
[38]
Automated data denoising for recommendation
Yingqiang Ge, Mostafa Rahmani, Athirai Irissappane, Jose Sepulveda, James Caverlee, and Fei Wang. Automated data denoising for recommendation. arXiv preprint arXiv:2305.07070, 2023
2023 arXiv
-
[39]
Dtec: Dual training error based correction approach for recommender systems
Costas Panagiotakis, Harris Papadakis, Antonis Papagrigoriou, and Paraskevi Fragopoulou. Dtec: Dual training error based correction approach for recommender systems. Software Impacts, 9:100111, 2021
2021
-
[40]
Improving recommendation quality through outlier removal
Yuan-Yuan Xu, Shen-Ming Gu, and Fan Min. Improving recommendation quality through outlier removal. International Journal of Machine Learning and Cybernetics, 13(7):1819–1832, 2022
2022
-
[41]
A semantic approach to remove incoherent items from a user profile and improve the accuracy of a recommender system
Roberto Saia, Ludovico Boratto, and Salvatore Carta. A semantic approach to remove incoherent items from a user profile and improve the accuracy of a recommender system. Journal of Intelligent Information Systems, 47:111–134, 2016
2016
-
[42]
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019
2019
-
[43]
Bars: Towards open benchmarking for recommender systems
Jieming Zhu, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Xi Xiao, and Rui Zhang. Bars: Towards open benchmarking for recommender systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 29...
2022
-
[44]
C Additional Experiments C.1 Numerical Experiments Figure 6 illustrate how both optimizers perform in a noise-free environment
with Lemma B.2 and Lemma B.3, respectively. C Additional Experiments C.1 Numerical Experiments Figure 6 illustrate how both optimizers perform in a noise-free environment. Figure 6: Performance of Adam (top row) and CAdam (bottom row) on four different optimization landscapes ...
-
[45]
Separable L1 Loss: f1(x, y) = |x| + |y|
-
[46]
Inseparable L1 Loss: f2(x, y) = |x + y| + |x−y| 10
-
[47]
Inseparable L2 Loss: f3(x, y) = (x + y)2 + (x−y)2 10
-
[48]
Rosenbrock Function: f4(x, y) = (a − x)2 + b(y − x2)2, where a = 1 and b = 100. To simulate noise in the gradients, we applied a random mask to each dimension of the gradient with a 50% probability using the same random seed across different optimizers: ∇noisy(x, y) = ∇f (x, y...
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.