Improved constraints on Higgs boson self-couplings with quartic and cubic power dependencies of the cross section

Precise determination of the Higgs boson self-couplings is essential for understanding the mechanism underlying electroweak symmetry breaking. However, owing to the limited number of Higgs boson pair events at the LHC, only loose constraints have been established to date. Current constraints are based on the assumption that the cross section is a quadratic function of the trilinear Higgs self-coupling within the $\kappa$ framework. Incorporating higher-order quantum corrections from virtual Higgs bosons would significantly alter this functional form, introducing new quartic and cubic power dependencies on the trilinear Higgs self-coupling. To derive this new functional form, we propose a specialized renormalization procedure that tracks all Higgs self-couplings at each calculation step. Additionally, we introduce renormalization constants for coupling modifiers within the $\kappa$ framework to ensure the cancellation of all ultraviolet divergences. With new functional forms of the cross sections in both the gluon-gluon fusion and vector boson fusion channels, the upper limit of $\kappa_{\lambda_{\rm 3H}}=\lambda_{\rm 3H}/\lambda_{\rm 3H}^{\rm SM}$ set by the ATLAS (CMS) collaboration is reduced from 6.6 (6.49) to 5.5 (5.39). However, extracting a meaningful constraint on the quartic Higgs self-coupling $\lambda_{\rm 4H}$ from Higgs boson pair production data remains challenging. We also present the invariant mass distributions of the Higgs boson pair at different values of $\kappa_{\lambda}$, which could assist in setting optimal cuts for experimental analysis.