Searches for the BSM scenarios at the LHC using decision tree based machine learning algorithms: A comparative study and review of Random Forest, Adaboost, XGboost and LightGBM frameworks

Machine learning algorithms are now being extensively used in our daily lives, spanning across diverse industries as well as academia. In the field of high energy physics (HEP), the most common and challenging task is separating a rare signal from a much larger background. The boosted decision tree (BDT) algorithm has been a cornerstone of the high energy physics for analyzing event triggering, particle identification, jet tagging, object reconstruction, event classification, and other related tasks for quite some time. This article presents a comprehensive overview of research conducted by both HEP experimental and phenomenological groups that utilize decision tree algorithms in the context of the Standard Model and Supersymmetry (SUSY). We also summarize the basic concept of machine learning and decision tree algorithm along with the working principle of \texttt{Random Forest}, \texttt{AdaBoost} and two gradient boosting frameworks, such as \texttt{XGBoost}, and \texttt{LightGBM}. Using a case study of electroweakino productions at the high luminosity LHC, we demonstrate how these algorithms lead to improvement in the search sensitivity compared to traditional cut-based methods in both compressed and non-compressed R-parity conserving SUSY scenarios. The effect of different hyperparameters and their optimization, feature importance study using SHapley values are also discussed in detail.