Pith. sign in

REVIEW 37 references

LO2: Microservice API Anomaly Dataset of Logs and Metrics

T0 review · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LO2 provides over 657,000 labeled log files and 45 million metric files from a production open-source OAuth2 microservice system, for anomaly detection research.

arxiv 2504.12067 v1 pith:UNEBWR32 submitted 2025-04-16 cs.SE cs.DCcs.NI

Alexander Bakhtin , Jesse Nyyssölä , Yuqing Wang , Noman Ahmad , Ke Ping , Matteo Esposito , Mika Mäntylä , Davide Taibi This is my paper · ORCID
classification cs.SEcs.DCcs.NI
keywords datametricslogsanomalymicroservicesystemsystemsdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The LO2 dataset comes from Light-OAuth2, an open-source implementation of the OAuth2 login protocol built from seven microservices. The authors deployed the system with Docker, then used the Locust load-testing tool to send both correct API calls and deliberately broken calls for dozens of documented error cases. For every test they saved the log files produced by each service, plus host and container metrics collected by Prometheus. In total the raw data occupies about 540 GB and includes more than two billion log lines and 485 distinct metrics.

Each test is labeled either correct or as one of 53 specific errors, such as a missing password or a non-existent user. The authors also publish a replication package so others can run the same collection procedure. They report an initial check: a decision tree can detect error tests from service logs with F1 scores around 0.72 when services are pooled, and much higher when one error type is viewed at a time. Principal component analysis of the metrics mostly highlighted memory-related host metrics.

A notable gap is traces. The authors injected Jaeger, but the Light-OAuth2 services were not instrumented to emit meaningful traces, so most trace files are empty or single spans. The paper is honest about this in the body, though the abstract still says the dataset covers traces. That mismatch, plus a small counting error in the analysis description, are the main blemishes.

Extended reading notes

Core claim

The paper's central contribution is the dataset itself: 'We provide a dataset of monitoring data for a production-ready (non-demo) MSS' with logs and metrics 'labeled according to the error triggered during the call' (Section 1, Contributions). If the paper is correct, the community gains a large, reproducible benchmark for log and metric anomaly detection in an open-source OAuth2 microservice system, covering 53 API error types.

Load-bearing premise

The dataset's labels are assigned at the test level, not the request level: during each error test the Locust suite runs correct tasks alongside the single erroneous task (Section 3.3, step 5a), and the whole service log file for that test is then labeled as that error. The assumption is that the difference between an error-test log and a correct-test log is attributable to the introduced error rather than to incidental task mix or ordering. If this fails, anomaly detectors trained on these labels may learn test-composition artifacts instead of the target error.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central contribution is a dataset, not a mathematical derivation. No numeric constants are fitted to make the dataset exist, and no new entities are postulated. The claims rest on domain assumptions about the test environment and labeling, plus standard statistics; hand-chosen analysis thresholds affect only the preliminary results.

free parameters (5)
  • PCA top-3 component count = 3
    Hand-chosen number of principal components used to summarize metric variance in Section 5.2. The choice is arbitrary and affects the list of important metrics, but not the dataset itself.
  • PCA top-5 features per component = 5
    Hand-chosen cutoff for listing influential features per component. Arbitrary and only affects the preliminary metric analysis.
  • Log analysis sample size = 100 runs out of 1,740
    The preliminary anomaly detection uses a subset of 100 runs; results in Tables 3 and 4 depend on this sample and on random selection of one error test per run.
  • Train/test split ratio = 50-50
    The log anomaly detection experiments split data evenly between training and testing; a standard but arbitrary choice reported in Section 5.1.
  • Locust task weights = equal weights
    All correct and erroneous Locust tasks had equal probability of being selected at runtime, as stated in Sections 7 and 10. This shapes the request distribution in the dataset and is acknowledged as unrealistic.
assumptions (4)
  • domain assumption Docker container logs and Prometheus node exporter metrics are sufficient to observe the API errors under study.
    The dataset's value for anomaly detection relies on this; Section 3.4 describes collection, Section 5 gives indirect evidence via model scores, but there is no external validation that the signals isolate the specific errors.
  • domain assumption A service log file generated during an error test can be treated as a single anomalous sequence, even though correct and error tasks run together in that test.
    Section 3.3 step 5a runs 'correct + that particular error task' and Section 4.1 labels files at the run/test level. Request-level attribution is only possible through the separately provided Locust logs.
  • domain assumption Light-OAuth2, deployed with its standard Docker Compose and an arbitrary MySQL choice, is representative enough of production microservice systems for generalizable research conclusions.
    Stated as the system of study in Section 3.1; external validity limits are acknowledged in Section 10.
  • standard math Standard statistical tools (PCA, F1-score, AUCROC) are appropriate for the preliminary analysis.
    Used without further justification in Section 5; standard practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LO2: Microservice API Anomaly Dataset of Logs and Metrics." pith.science (2026). https://pith.science/paper/UNEBWR32

@misc{pith2026250412067,
  author       = {Pith},
  title        = {Pith review of: LO2: Microservice API Anomaly Dataset of Logs and Metrics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UNEBWR32}},
  note         = {Machine review of arXiv:2504.12067}
}
read the original abstract

Context. Microservice-based systems have gained significant attention over the past years. A critical factor for understanding and analyzing the behavior of these systems is the collection of monitoring data such as logs, metrics, and traces. These data modalities can be used for anomaly detection and root cause analysis of failures. In particular, multi-modal methods utilizing several types of this data at once have gained traction in the research community since these three modalities capture different dimensions of system behavior. Aim. We provide a dataset that supports research on anomaly detection and architectural degradation in microservice systems. We generate a comprehensive dataset of logs, metrics, and traces from a production microservice system to enable the exploration of multi-modal fusion methods that integrate multiple data modalities. Method. We dynamically tested the various APIs of the MS-based system, implementing the OAuth2.0 protocol using the Locust tool. For each execution of the prepared test suite, we collect logs and performance metrics for correct and erroneous calls with data labeled according to the error triggered during the call. Contributions. We collected approximately 657,000 individual log files, totaling over two billion log lines. In addition, we collected more than 45 million individual metric files that contain 485 unique metrics. We provide an initial analysis of logs, identify key metrics through PCA, and discuss challenges in collecting traces for this system. Moreover, we highlight the possibilities for making a more fine-grained version of the data set. This work advances anomaly detection in microservice systems using multiple data sources.

Figures

Figures reproduced from arXiv: 2504.12067 by the authors.

Figure 1
Figure 1. Data Collection Process DEBUG Stacktr. ERROR Exception INFO Other WARN Log Type 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Count 1e8 Services client code key refresh-token service token user [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Number of Log Lines per Type and Service [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 27 canonical work pages

  1. [1]

    Roberto Amadini, Simone Gazza, Jacopo Soldani, Monica Vitali, Antonio Brogi, Stefano Forti, Saverio Giallorenzo, Pierluigi Plebani, Francisco Ponce, and Gi- anluigi Zavattaro. 2024. Pick a Flavour: Towards Sustainable Deployment of Cloud-Edge Applications. In International Symposium on Logic-Based Program Synthesis and Transformation. Springer, 117–127

    work page 2024

  2. [2]

    Arlitt and T

    M. Arlitt and T. Jin. 2000. A workload characterization study of the 1998 World Cup Web site. IEEE Network 14, 3 (2000), 30–37. doi:10.1109/65.844498

    work page doi:10.1109/65.844498 2000

  3. [3]

    Alexander Bakhtin, Matteo Esposito, Valentina Lenarduzzi, and Davide Taibi

    work page

  4. [4]

    Alexander Bakhtin, Jesse Nyyssölä, Yuqing Wang, Noman Ahmad, Ke Ping, Matteo Esposito, Mika Mäntylä, and Davide Taibi. 2024. LO2: Microservice Dataset of Logs and Metrics [Data set]. doi:10.5281/zenodo.14257989

    work page doi:10.5281/zenodo.14257989 2024

  5. [5]

    LO2: Microservice Dataset of Logs and Metrics

    Alexander Bakhtin, Jesse Nyyssölä, Yuqing Wang, Noman Ahmad, Ke Ping, Matteo Esposito, Mika Mäntylä, and Davide Taibi. 2024. Replication for "LO2: Microservice Dataset of Logs and Metrics". doi:10.5281/zenodo.14229369

    work page doi:10.5281/zenodo.14229369 2024

  6. [6]

    Creative Commons. 2024. Creative Commons Attribution 4.0 International Public License. https://creativecommons.org/licenses/by/4.0/ Accessed: 2024-10-29

    work page 2024

  7. [7]

    Matteo Esposito and Davide Falessi. 2024. VALIDATE: A deep dive into vulnera- bility prediction datasets. Information and Software Technology (2024), 107448

    work page 2024

  8. [8]

    Davide Falessi, Simone Mesiano Laureani, Jonida Çarka, Matteo Esposito, and Daniel Alencar da Costa. 2023. Enhancing the defectiveness prediction of methods and classes via JIT. Empir. Softw. Eng. 28, 2 (2023), 37. doi:10.1007/S10664-022- 10261-Z

    work page doi:10.1007/s10664-022- 2023

Show all 37 references
  1. [9]

    FORCE11. 2014. The FAIR Data Principles. https://force11.org/info/the-fair- data-principles/ Accessed: 2024-12-02

    2014

  2. [10]

    FudanSELab. 2024. DeepTraLog. https://github.com/FudanSELab/DeepTraLog

    2024

  3. [11]

    Sona Ghahremani. 2021. 1998 World Cup Website Access Logs. doi:10.5281/zenodo. 5145855

    2021 doi

  4. [12]

    D. Hardt. 2012. The OAuth 2.0 Authorization Framework. Request for Comments

    2012

  5. [13]

    Andre Hora. 2024. Test polarity: detecting positive and negative tests. In Com- panion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 537–541

    2024

  6. [14]

    IntelligentDDS. 2024. Nezha. https://github.com/IntelligentDDS/Nezha

    2024

  7. [15]

    Max Landauer, Florian Skopik, and Markus Wurzenberger. 2024. A Critical Review of Common Log Data Sets Used for Evaluation of Sequence-Based Anomaly Detection Techniques. Proc. ACM Softw. Eng. 1, FSE, Article 61 (July 2024), 22 pages. doi:10.1145/3660768

    2024 doi

  8. [16]

    Lawrence Berkeley National Laboratory. 1998. 1998 World Cup Dataset. https: //ita.ee.lbl.gov/html/contrib/WorldCup.html Accessed: 2024-10-29

    1998

  9. [17]

    Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, and Michael R. Lyu. 2023. Eadro. doi:10.5281/zenodo.7615394 [Dataset]

    2023 doi

  10. [18]

    Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, and Michael R. Lyu. 2023. Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi- source Data. In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2...

    2023

  11. [19]

    Valentina Lenarduzzi, Nyyti Saarimäki, and Davide Taibi. 2019. The Technical Debt Dataset. In Conference on Predictive Models and Data Analytics in Software Engineering

    2019

  12. [20]

    Zeyan Li, Nengwen Zhao, Mingjie Li, Xianglin Lu, Lixin Wang, Dongdong Chang, Xiaohui Nie, Li Cao, Wenchi Zhang, Kaixin Sui, et al . 2022. Actionable and interpretable fault localization for recurring failures in online service systems. In Proceedings of the 30th ACM Joint Euro...

    2022

  13. [21]

    Mika V Mäntylä, Yuqing Wang, and Jesse Nyyssölä. 2024. Loglead-fast and integrated log loader, enhancer, and anomaly detector. In 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 395–399

    2024

  14. [22]

    NetManAIOps. 2022. DéjàVu. doi:10.5281/zenodo.6955909 [Dataset]

    2022 doi

  15. [23]

    Jesse Nyyssölä and Mika Mäntylä. 2024. Speed and Performance of Parserless and Unsupervised Anomaly Detection Methods on Software Logs. In 2024 IEEE 24th International Conference on Software Quality, Reliability and Security (QRS) . 657–666. doi:10.1109/QRS62785.2024.00071

    2024

  16. [24]

    Joshua Owotogbe, Indika Kumara, Willem-Jan Van Den Heuvel, and Damian An- drew Tamburri. 2024. Chaos Engineering: A Multi-Vocal Literature Review. arXiv:2412.01416 [cs.SE] https://arxiv.org/abs/2412.01416

    2024 arXiv

  17. [25]

    Luan Pham. 2024. RCAEval: A Benchmark for Root Cause Analysis of Microser- vice Systems. doi:10.5281/ZENODO.14590730

    2024 doi

  18. [26]

    Simon Schneider and Riccardo Scandariato. 2023. Automatic extraction of security-rich dataflow diagrams for microservice applications written in Java. Journal of Systems and Software 202 (2023), 111722

    2023

  19. [27]

    Murugiah Souppaya, John Morello, and Karen Scarfone. 2017. Application Con- tainer Security Guide. Technical Report SP 800-190. National Institute of Standards and Technology (NIST). https://nvlpubs.nist.gov/nistpubs/SpecialPublications/ NIST.SP.800-190.pdf

    2017

  20. [28]

    Davide Taibi, Valentina Lenarduzzi, and Claus Pahl. 2017. Processes, motivations, and issues for migrating to microservices architectures: An empirical investiga- tion. IEEE Cloud Computing 4, 5 (2017), 22–32

    2017

  21. [29]

    Tsinghua University. 2021. AIOps Challenge 2021. https://www.aiops.cn/gitlab/ aiops-nankai/data/trace/aiops2021

    2021

  22. [30]

    Ohlsson, and Björn Regnell

    Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, and Björn Regnell

  23. [31]

    Guangba Yu, Pengfei Chen, Yufeng Li, Hongyang Chen, Xiaoyun Li, and Zibin Zheng. 2023. Nezha: Interpretable fine-grained root causes analysis for microser- vices on multi-modal observability data. In Proceedings of the 31st ACM Joint European Software Engineering Conference an...

    2023

  24. [32]

    Chenxi Zhang, Xin Peng, Chaofeng Sha, Ke Zhang, Zhenqing Fu, Xiya Wu, Qing- wei Lin, and Dongmei Zhang. 2022. Deeptralog: Trace-log combined microservice anomaly detection through graph-based deep learning. In Proceedings of the 44th international conference on software engine...

    2022

  25. [33]

    Jieming Zhu, Shilin He, Pinjia He, Jinyang Liu, and Michael R. Lyu. 2023. Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) . 355–366. doi:10.1109/ISSRE59848.2023.00071

    2023

  26. [34]

    Yuan Zuo, Yulei Wu, Geyong Min, Chengqiang Huang, and Ke Pei. 2020. An Intelligent Anomaly Detection Scheme for Micro-Services Architectures With Temporal and Spatial Data Analysis. IEEE Transactions on Cognitive Communica- tions and Networking 6, 2 (2020), 548–561. doi:10.110...

    2020

  27. [2012]

    Springer

    Experimentation in Software Engineering . Springer

  28. [2025]

    In 2025 IEEE 22nd International Conference on Software Architecture (ICSA)

    Network Centrality as a New Perspective on Microservice Architecture. In 2025 IEEE 22nd International Conference on Software Architecture (ICSA) . doi:10. 1109/ICSA65012.2025.00017

    2025

  29. [6749]

    https://www.rfc-editor.org/rfc/rfc6749

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.