Pith. sign in

REVIEW 4 major objections 7 minor 64 references

Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Continual pre-training on natural-language Q&A pairs about logs, instead of raw log text, is claimed to adapt open-source LLMs to log analysis and to yield the best reported results across four tasks.

desk verdict Worth a serious look for the NLPLog resource and the interpretable-vs-raw CPT comparison, but the headline accuracy claims are undermined by likely train/test overlap and by LLM-referenced unseen-domain evaluation. read the letter →

arxiv 2412.01377 v2 pith:DLLN3PK3 submitted 2024-12-02 cs.CL cs.SE

Yuhe Ji , Yilun Liu , Feiyu Yao , Minggui He , Shimin Tao , Xiaofeng Zhao , Su Chang , Xinhua Yang
show 5 more authors
Weibin Meng Yuming Xie Boxing Chen Shenglin Zhang Yongqian Sun
This is my paper · ORCID
classification cs.CLcs.SE
keywords loganalysiscontinualpre-traininglargelanguagemodelsdomainadaptationinterpretabilityquestion-answerdatasetparsinganomalydetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the right way to adapt a general-purpose open-source LLM to log analysis is to continue pre-training it on natural-language question-answer pairs that explain logs, rather than on raw log lines. To do this the authors build NLPLog, a dataset of over 250,000 Q&A pairs covering five dimensions of log knowledge across 14 domains, and use it to continual-pretrain LLaMA2-7B into SuperLog. They report that SuperLog outperforms prior methods on log parsing, anomaly detection, fault diagnosis, and log interpretation, with an average improvement of 12.01% over the second-best baseline. An ablation replacing the Q&A corpus with raw logs gives about a 23% lower average performance, supporting the claim that interpretable knowledge, not just domain text, drives the gain.

What carries the argument

The load-bearing object is NLPLog, a corpus of 250,000+ Q&A pairs in natural language, each pairing a real log event with answers along five knowledge dimensions: Grok pattern parsing, log event insights, root cause analysis, component correlation, and potential failure forecast. The construction pipeline starts from LogHub logs, extracts 51,590 unique templates with LogPPT, reconstructs representative events by recombining templates with variable groups, and generates the Q&A pairs with a large proprietary LLM followed by human calibration. Continual pre-training then runs LLaMA2-7B on these pairs for 1.5 epochs at a learning rate of 1e-5, followed by task-specific or general instruction tuning.

What would settle it

Take the 51,590 NLPLog templates and the evaluation templates from LogHub_2k and BGL/Spirit, and run exact and near-duplicate matching after masking variables. If a large fraction of the test templates or their paraphrases appear in the continual-pretraining corpus, the 12.01% average gain is explained by memorization rather than by domain adaptation; in that case, retraining on domains entirely absent from the evaluation set would show whether the parsing and anomaly-detection gains persist.

Watch

Extended reading notes

Core claim

SuperLog is a LLaMA2-7B model that first undergoes continual pre-training on NLPLog, a dataset of over 250,000 natural-language question-answer pairs built from real logs, and then lightweight instruction tuning. The paper's discovery is that this order, interpretable Q&A before task fine-tuning, transfers log knowledge without the distribution mismatch and catastrophic forgetting observed when continual pre-training is done on raw logs. The authors report the best performance on log parsing, anomaly detection, failure diagnosis, and log interpretation, with an average 12.01% improvement over the second-best model and strong outputs on Apache and OpenStack logs not seen during training.

Load-bearing premise

The reported gains assume that NLPLog's training entries are disjoint from the logs and templates used in evaluation, since NLPLog is built from the same LogHub domains (HDFS, Hadoop, Zookeeper, Linux, Proxifier, BGL) that appear in the parsing and anomaly-detection tests and no overlap check is reported.

Editorial extensions

If this is right

  • If the central claim holds, continual pre-training on interpretable Q&A pairs is a general recipe for adapting open-source LLMs to log analysis, replacing raw-log continual pre-training.
  • Log interpretation quality improves under Q&A continual pre-training, whereas raw-log continual pre-training lowers it relative to no continual pre-training, meaning interpretability is not sacrificed for task accuracy.
  • Performance transfers to Apache and OpenStack logs outside the training domains, suggesting the learned knowledge is not limited to the 14 training domains.
  • The 23% average gain in the ablation over raw-log training quantifies the value of the interpretable-knowledge format itself, not just the extra data.
  • The 12.01% average improvement over the second-best model across four tasks implies that the method is competitive without relying on proprietary APIs at inference time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's evaluation domains overlap its pre-training domains, so the cleanest test of the generalization claim would be to pre-train on one set of log domains and evaluate on disjoint ones with human ground truth; the current unseen-domain test uses ROUGE against another LLM's outputs.
  • The same Q&A-injection scheme could be tried on other semi-structured operational data, such as network traces, Kubernetes events, or cloud billing records, where raw tokens are far from natural language.
  • Because the Q&A answers come from a proprietary LLM, the dataset's ceiling is set by that generator; a cheaper open loop would distill explanations from the base model itself or from retrieval over documentation.
  • A direct comparison of continual-pretraining token budgets would disentangle 'more data helps' from 'Q&A format helps'; the ablation holds the data source fixed but does not necessarily hold the token count fixed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes SuperLog, a LLaMA-2-7B model adapted to log analysis by continual pre-training (CPT) on NLPLog, a corpus of over 250,000 natural-language question-answer pairs distilled from LogHub logs by a proprietary LLM. The authors evaluate SuperLog on log parsing, anomaly detection, failure diagnosis/interpretation, and logs from unseen domains, reporting state-of-the-art results and an ablation that supports interpretable-knowledge CPT over raw-log CPT. The manuscript claims an average accuracy improvement of 12.01% over the second-best model and a 23% average improvement over raw-log training.

Significance. If the empirical results were valid, the paper would provide a practical and reusable recipe for adapting open-source LLMs to log analysis, and NLPLog would be a valuable community resource. The core idea of replacing raw-log CPT with interpretable natural-language Q&A is interesting and the dataset release is a concrete contribution. However, the validity of the empirical claims is not established: the training corpus overlaps in domain and likely in templates with the main evaluation benchmarks, the unseen-domain evaluation is a self-agreement check against an LLM-generated reference, and several reported numbers are internally inconsistent. The potential significance is therefore conditional on a clean, leakage-free evaluation.

major comments (4)
  1. [§3.1, Table 1; §4.4.1, Tables 2-3] The log-parsing and anomaly-detection evaluations are not protected against training/evaluation overlap. NLPLog is constructed from templates extracted from the full LogHub domains (Section 3.1.1), including HDFS, Hadoop, Zookeeper, Linux, Proxifier, and BGL (Table 1). The log-parsing test set is the last 90% of LogHub_2k for the same five domains, and the anomaly-detection test uses BGL template-label pairs (Section 4.4.1). The paper reports no check for exact template matches, near-duplicates, or paraphrases between NLPLog training entries and the test logs/templates. Because NLPLog deduplication only removes duplicate templates within the CPT corpus, it cannot prevent test templates from being present in training. If test templates were seen during CPT, parsing F1 values near 0.99 and the anomaly-detection gains reflect memorization rather than a generalizable domain-adaptation advantage. The same problem affects the w/o IK ablation in Section 4.5.1, which trains on raw logs from the same domains, so Table 6 does not establish generalization beyond memorized templates. Please report exact overlap statistics at template and event level, and re-run all affected experiments after excluding overlapping entries, or evaluate on domains disjoint from NLPLog.
  2. [§4.6, Table 7; §5.2.1] The unseen-domain evaluation cannot support the claimed generalization. Table 7 measures ROUGE similarity to references produced by an unnamed 'advanced LLM' rather than to ground-truth templates or human annotations. The paper itself concedes in Section 5.2.1 that high surface similarity to these proxy answers does not guarantee that the model grasped the underlying task. A model that imitates the style of the reference LLM could score well on ROUGE without performing correct parsing. This self-agreement check is circular as evidence of capability, so the claim of 'exceptional performance on logs from unseen domains' is unsupported. Please evaluate on labeled data (e.g., human-annotated templates) or report task-specific accuracy against ground truth, and justify that the reference is a reliable gold standard.
  3. [§1 and §4.5, Table 6] The headline '23% average performance improvement' is not reproducible from the reported ablation. Table 6 reports log interpretation as 3.895 for SuperLog, but Table 4 gives usefulness 4.430 and readability 4.780, whose average is 4.605, and Section 4.5.2 states that the interpretation score is the average of usefulness and readability. In addition, relative improvements of SuperLog over w/o IK computed from Table 6 are approximately 1.5% for parsing, 21.9% for anomaly detection, 30.9% for diagnosis, and 27.5% for interpretation, whose average is about 20.5%, not 23%. Please clarify the exact aggregation formula, reconcile the two tables, and state precisely how the 23% and 12.01% figures are computed.
  4. [Tables 2-7] All comparisons are based on single runs; no standard deviations, confidence intervals, or significance tests are reported. This matters because several differences are small (e.g., BGL session-level anomaly detection F1: 0.147 for SuperLog vs. 0.129 for LogPrompt in Table 3), and few-shot fine-tuning on a few thousand examples is stochastic. Please report multiple seeds and statistical tests, or explicitly justify that the reported differences exceed run-to-run noise.
minor comments (7)
  1. [Abstract and §1] There are typos such as 'pathes' in the abstract, 'trainging' and 'propietary' in Section 1, and 'decipted' in Section 1.
  2. [Table 2] Several entries lack separating spaces, e.g., '0.9480.889' for LogPrompt and '0.9980.815' for SuperLog; these should be fixed for readability.
  3. [§4.1.2 and §4.5.1] The dataset name is inconsistent: 'AlpaCar_1k' in Section 4.1.2 versus 'Alpaca-1k' in Section 4.5.1.
  4. [§3.1.3 and §4.6.1] The 'state-of-the-art LLM' used for knowledge generation and the 'advanced LLM' used for unseen-domain references are never named; please identify the model and version for reproducibility.
  5. [§3.1.3] The 'human-calibration phase' is described only qualitatively; please report how many responses were filtered, by how many annotators, and the inter-annotator agreement.
  6. [§4.4.2] The log-interpretation evaluation uses four experts on 100 logs but reports no inter-annotator agreement; a mean score without agreement information is difficult to interpret.
  7. [Abstract and §5.1] The phrase 'average accuracy improvement of 12.01%' is misleading because the metrics are RI, F1, ROUGE, and human scores rather than accuracy; please use task-appropriate terminology.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: the unseen-domain evaluation is self-referential (LLM-generated reference vs LLM-distilled training data), and the parsing/anomaly benchmarks overlap the NLPLog CPT domains with no reported overlap check; the core four-task numbers still rely on human-labeled data.

  1. fitted input called prediction [Section 3.1.1 (NLPLog construction from LogHub) vs Section 4.4.1 (log parsing/anomaly benchmarks on LogHub_2k and BGL)]
    "To construct NLPLog dataset, we choose 14 different log domains from LogHub [19]... After applying the log template extraction algorithm, we divided the logs into their template and variable components. Duplicate log templates were eliminated, resulting in 51,590 distinct log templates... This benchmark assesses the performance of log parsing on the last 90% of log entries from five distinct domains within the LogHub_2k dataset."

    NLPLog is built from all logs in LogHub domains including exactly the domains used for evaluation: Table 1 lists HDFS, Hadoop, Zookeeper, Linux, Proxifier and BGL, and Section 4.4.1 tests log parsing on LogHub_2k for HDFS, Hadoop, Zookeeper, Linux, Proxifier and anomaly detection on BGL templates. Since LogHub_2k is sampled from the same LogHub corpus, the template of each test log was almost certainly one of the 51,590 templates used to generate NLPLog Q&A pairs. The paper reports no overlap/deduplication check between the CPT corpus and the test logs. The parsing/anomaly 'predictions' are therefore partly recall of training templates, so the reported gains are not by construction evidence of domain-adaptation generalization.

  2. other [Section 4.6.1 (unseen-domain evaluation) vs Section 3.1.3 (LLM-generated training answers) and Section 5.2.1]
    "Since these datasets do not have ground truth labels, we compare the model’s output with results generated by an advanced LLM to serve as a reference for evaluation. Specifically, we replicate the log parsing experiment setup used in previous studies, where results generated by an advanced LLM for Apache and OpenStack logs are treated as the target labels."

    The RQ3 claim of strong unseen-domain performance is scored by ROUGE against an LLM-generated reference, but the NLPLog training answers were themselves generated by 'a state-of-the-art LLM' (Section 3.1.3), with only a human-calibration filter. The evaluation therefore measures agreement with the same teacher distribution the model was distilled from, not independent ground truth. Section 5.2.1 concedes: 'high surface similarity to these proxy answers does not guarantee the method has adequately grasped the underlying task.' This makes the unseen-domain 'prediction' self-referential rather than externally validated.

full rationale

The paper's central construction—building NLPLog from LogHub templates and continual pre-training LLaMA2—does not by itself define the evaluation outcomes; the headline 12.01% improvement is an empirical result, and the four-task benchmarks mostly use human-annotated labels, so the mathematical derivation is not circular. The main circularity risks are empirical rather than formal. First, the CPT corpus is constructed from the same LogHub domains used for the log parsing and anomaly detection tests, including HDFS, Hadoop, Zookeeper, Linux, Proxifier, and BGL, and the paper reports no overlap or deduplication check; because LogHub_2k is drawn from LogHub, the test templates were plausibly present in the training Q&A pairs, making the parsing and anomaly-detection 'predictions' partly memorization of training templates. Second, the RQ3 unseen-domain evaluation treats LLM-generated outputs as target labels while the training data itself was generated by a state-of-the-art LLM, so the evaluation measures agreement with the teacher distribution rather than independent correctness; the authors explicitly acknowledge this proxy limitation in Section 5.2.1. These two issues make parts of the generalization evidence self-referential, but they do not reduce the entire derivation to its inputs, so a moderate score of 4 is appropriate.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The central claim rests on the quality of the LLM-generated training data, the correctness of the template-extraction tool, the disjointness of training and evaluation data, and the validity of using an LLM as a reference for unseen-domain evaluation.

assumptions (4)
  • domain assumption LogPPT template extraction yields correct templates from LogHub logs when trained on 2,000 manually parsed entries per domain.
    Section 3.1.1 uses LogPPT to deduplicate logs and reconstruct events; parsing errors would propagate into NLPLog.
  • domain assumption The unnamed 'state-of-the-art LLM' produces correct, comprehensive log-knowledge answers, and the human-calibration phase removes inaccurate answers.
    Section 3.1.3 generates all Q&A pairs with a proprietary LLM and filters via an unspecified calibration step; no inter-annotator agreement or error rate is reported.
  • domain assumption The first 10% of logs used for few-shot fine-tuning do not overlap with the 90% test portion, and test logs were not seen during CPT.
    Section 4.1.1 splits by position; the paper does not check overlap between the CPT corpus, built from all logs of the same domains, and the test logs.
  • domain assumption ROUGE scores against 'an advanced LLM' output are a valid proxy for log-parsing quality on label-free domains.
    Section 4.6.1 uses LLM-generated references as target labels; Section 5.2.1 admits high surface similarity does not guarantee task understanding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge." pith.science (2026). https://pith.science/paper/DLLN3PK3

@misc{pith2026241201377,
  author       = {Pith},
  title        = {Pith review of: Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DLLN3PK3}},
  note         = {Machine review of arXiv:2412.01377}
}
read the original abstract

Log analysis represents a critical sub-domain within AI applications that facilitates automatic approaches to fault and error management of large-scaled software systems, saving labors of traditional manual methods. While existing solutions using large language models (LLMs) show promise, they are limited by a significant domain gap between natural and log languages (the latter contains rich domain-specific tokens such as status codes, IP addresses, resource pathes), which restricts their effectiveness in real-world applications. However, directly adapting general-purpose LLMs to log analysis using raw logs may degrade their performance due to inconsistent token distribution. In this paper, we present a domain adaptation approach that addresses these limitations by integrating interpretable domain knowledge into open-source LLMs through continual pre-training (CPT), which bridges this domain gap by adapting LLMs on interpretable natural texts with log knowledge (instead of raw logs) to reduce distribution discrepancy. To achieve this, we developed NLPLog, a comprehensive dataset containing over 250,000 question-answer pairs on log-related knowledge. Our resulting model, SuperLog, achieves the best performance across four log analysis tasks, with an average accuracy improvement of 12.01% over the second-best model. Ablation study also suggests advantages of domain adaption using interpretable log knowledge over using raw logs.

Figures

Figures reproduced from arXiv: 2412.01377 by the authors.

Figure 1
Figure 1. Illustration on differences of three LLM-based log [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration on the interpretable knowledge con [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 44 canonical work pages

  1. [1]

    J. Bai, S. Bai, Y. Chu, et al . 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 (2023)

    arXiv 2023

  2. [2]

    Beijing Academy of Artificial Intelligence. 2023. Aquilachat. https://model.baai. ac.cn/model-detail/100101

    work page 2023

  3. [3]

    Z. Cai, M. Cao, H. Chen, et al. 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297 (2024)

    arXiv 2024

  4. [4]

    Y. Chen, H. Xie, M. Ma, et al. 2024. Automatic Root Cause Analysis via Large Language Models for Cloud Incidents. In Proc. of the European Conference on Computer Systems

    work page 2024

  5. [5]

    Chiang, Z

    W.L. Chiang, Z. Li, Z. Lin, et al. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. See https://vicuna. lmsys. org (2023)

    work page 2023

  6. [6]

    T. Cui, S. Ma, Z. Chen, et al. 2024. LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis. arXiv preprint arXiv:2407.01896 (2024)

    arXiv 2024

  7. [7]

    Debnath, M

    B. Debnath, M. Solaimani, M.A.G. Gulzar, et al . 2018. LogLens: A Real-Time Log Analysis System. In Proc. of the IEEE International Conference on Distributed Computing Systems (ICDCS). CIKM ’25, November 10–14, 2025, Seoul, Republic of Korea Ji, Liu, et al

    work page 2018

  8. [8]

    J. Devlin. 2018. Bert: Pre-training of Deep Bidirectional Transformers for Lan- guage Understanding. arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

Show all 64 references
  1. [9]

    Du and F

    M. Du and F. Li. 2016. Spell: Streaming Parsing of System Event Logs. In Proc. of the IEEE International Conference on Data Mining (ICDM)

    2016

  2. [10]

    M. Du, F. Li, G. Zheng, and V. Srikumar. 2017. Deeplog: Anomaly Detection and Diagnosis from System Logs Through Deep Learning. InProc. of the ACM SIGSAC Conference on Computer and Communications Security

    2017

  3. [11]

    Ebert, G

    C. Ebert, G. Gallardo, J. Hernantes, and N. Serrano. 2016. DevOps. IEEE software (2016)

    2016

  4. [12]

    Egersdoerfer, D

    C. Egersdoerfer, D. Zhang, and D. Dai. 2023. Early Exploration of Using ChatGPT for Log-based Anomaly Detection on Parallel File Systems Logs. In Proc. of the 32nd International Symposium on High-Performance Parallel and Distributed Computing

    2023

  5. [13]

    Q. Fu, J.G. Lou, Y. Wang, and J. Li. 2009. Execution Anomaly Detection in Distributed Systems Through Unstructured Log Analysis. In Proc. of the IEEE International Conference on Data Mining

    2009

  6. [14]

    Y. Ge, Y. Liu, C. Hu, et al. 2024. Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation. In Proc. of the Conference on Empirical Methods in Natural Language Processing

    2024

  7. [15]

    H. Guo, J. Yang, J. Liu, et al. 2024. OWL: A Large Language Model for IT Opera- tions. (2024)

    2024

  8. [16]

    H. Guo, S. Yuan, and X. Wu. 2021. Logbert: Log Anomaly Detection via Bert. In Proc. of the International Joint Conference on Neural Networks (IJCNN)

    2021

  9. [17]

    Gururangan, A

    S. Gururangan, A. Marasović, S. Swayamdipta, et al. 2020. Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks. arXiv preprint arXiv:2004.10964 (2020)

    2020 arXiv

  10. [18]

    P. He, J. Zhu, Z. Zheng, and M.R. Lyu. 2017. Drain: An Online Log Parsing Approach with Fixed Depth Tree. In Proc. of the IEEE International Conference on Web Services (ICWS)

    2017

  11. [19]

    S. He, J. Zhu, P. He, and M.R. Lyu. 2020. Loghub: A Large Collection of System Log Datasets Towards Automated Log Analytics. arXiv preprint arXiv:2008.06448 (2020)

    2020 arXiv

  12. [20]

    Y. Huo, Y. Su, C. Lee, and M.R. Lyu. 2023. SemParser: A Semantic Parser for Log Analytics. In Proc. of the IEEE/ACM International Conference on Software Engineering (ICSE)

    2023

  13. [21]

    Jiang, A

    A.Q. Jiang, A. Sablayrolles, A. Mensch, et al . 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)

    2023 arXiv

  14. [22]

    Jiang, H

    Z. Jiang, H. Lin, Y. Zhong, et al. 2024. MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs. arXiv preprint arXiv:2402.15627 (2024)

    2024 arXiv

  15. [23]

    Jiang, J

    Z. Jiang, J. Liu, Z. Chen, et al. 2024. LILAC: Log Parsing Using LLMs with Adaptive Parsing Cache. Proc. of the ACM on Software Engineering (2024)

    2024

  16. [24]

    Jouppi, G

    N.P. Jouppi, G. Kurian, S. Li, et al. 2023. TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings. Proc. of the 50th Annual International Symposium on Computer Architecture(2023)

    2023

  17. [25]

    Jinhan Kim et al . 2020. Automatic abnormal log detection by analyzing log history for debugging insights. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Practice

    2020

  18. [26]

    Le and H

    V. Le and H. Zhang. 2021. Log-Based Anomaly Detection Without Log Parsing. In Proc. of the IEEE/ACM International Conference on Automated Software Engineering (ASE)

    2021

  19. [27]

    Le and H

    V. Le and H. Zhang. 2023. Log Parsing: How Far Can ChatGPT Go?. InProc. of the IEEE/ACM International Conference on Automated Software Engineering (ASE)

    2023

  20. [28]

    Le and H

    V. Le and H. Zhang. 2023. Log Parsing with Prompt-based Few-shot Learning. In Proc. of the IEEE/ACM International Conference on Software Engineering (ICSE)

    2023

  21. [29]

    Z. Li, C. Luo, T.H.P. Chen, et al . 2023. Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstraction. InICSE 2023

    2023

  22. [30]

    Y. Liu, Y. Ji, S. Tao, et al. 2024. LogLM: From Task-based to Instruction-based Automated Log Analysis. arXiv preprint arXiv:2410.09352 (2024)

    2024 arXiv

  23. [31]

    Y. Liu, S. Tao, W. Meng, et al. 2024. Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies. In Proc. of the IEEE/ACM International Conference on Program Comprehension

    2024

  24. [32]

    Y. Liu, S. Tao, W. Meng, et al. 2024. Logprompt: Prompt Engineering Towards Zero-Shot and Interpretable Log Analysis. In Proc. of the IEEE/ACM International Conference on Software Engineering: Companion Proceedings

    2024

  25. [33]

    Y. Luo, Z. Yang, F. Meng, et al. 2023. An Empirical Study of Catastrophic Forget- ting in Large Language Models During Continual Fine-Tuning. arXiv preprint arXiv:2308.08747 (2023)

    2023 arXiv

  26. [34]

    Z. Ma, A.R. Chen, D.J. Kim, et al. 2024. LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing. In Proc. of the IEEE/ACM International Conference on Software Engineering (ICSE)

    2024

  27. [35]

    Makanju, A.N

    A.A.O. Makanju, A.N. Zincir-Heywood, and E.E. Milios. 2009. Clustering Event Logs Using Iterative Partitioning. In Proc. of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

    2009

  28. [36]

    W. Meng, Y. Liu, F. Zaiter, et al. 2020. Logparse: Making Log Parsing Adaptive Through Word Classification. InProc. of the International Conference on Computer Communications and Networks (ICCCN)

    2020

  29. [37]

    W. Meng, Y. Liu, Y. Zhu, et al. 2019. LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs. In IJCAI

    2019

  30. [38]

    Messaoudi, A

    S. Messaoudi, A. Panichella, D. Bianculli, et al. 2018. A Search-Based Approach for Accurate Identification of Log Message Formats. In Proc. of the IEEE/ACM International Conference on Program Comprehension (ICPC)

    2018

  31. [39]

    Narayanan, M

    D. Narayanan, M. Shoeybi, J. Casper, et al. 2021. Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM. InProc. of the International Conference for High Performance Computing, Networking, Storage and Analysis

    2021

  32. [40]

    Oliner and J

    A. Oliner and J. Stearley. 2007. What Supercomputers Say: A Study of Five System Logs. In Proc. of the IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)

    2007

  33. [41]

    Pan, W.S

    J. Pan, W.S. Liang, and Y. Yidi. 2024. RAGLog: Log Anomaly Detection Using Retrieval Augmented Generation. In Proc. of the IEEE World Forum on Public Safety Technology (WFPST)

    2024

  34. [42]

    J. Qi, S. Huang, Z. Luan, et al . 2023. Loggpt: Exploring chatgpt for log-based anomaly detection. arXiv preprint arXiv:2309.01189 (2023)

    2023 arXiv

  35. [43]

    W.M. Rand. 1971. Objective Criteria for the Evaluation of Clustering Methods. J. Amer. Statist. Assoc. (1971)

    1971

  36. [44]

    Ruben Sipos and oters. 2014. Log-based predictive maintenance. In Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining. 1867–1876

    2014

  37. [45]

    Y. Sui, Y. Zhang, J. Sun, et al . 2023. LogKG: Log Failure Diagnosis Through Knowledge Graph. IEEE Transactions on Services Computing (2023)

    2023

  38. [46]

    X. Sun, X. Li, J. Li, et al. 2023. Text Classification Via Large Language Models. arXiv preprint arXiv:2305.08377 (2023)

    2023 arXiv

  39. [47]

    Ouyang Suriadi, Suriadi et al. 2013. Root cause analysis with enriched process logs. In Business Process Management Workshops: BPM 2012 International Workshops

    2013

  40. [48]

    L. Tang, T. Li, and C.S. Perng. 2011. LogSig: Generating System Events From Raw Textual Logs. In Proc. of the ACM International Conference on Information and Knowledge Management

    2011

  41. [49]

    S. Tao, Y. Liu, W. Meng, et al. 2023. Biglog: Unsupervised Large-scale Pre-training for a Unified Log Representation. In Proc. of the IEEE/ACM International Sympo- sium on Quality of Service (IWQoS)

    2023

  42. [50]

    S. Tao, W. Meng, Y. Cheng, et al. 2022. Logstamp: Automatic Online Log Parsing Based on Sequence Labelling. ACM SIGMETRICS Performance Evaluation Review (2022)

    2022

  43. [51]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, et al. 2023. Llama: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  44. [52]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, et al . 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  45. [53]

    Yuxia Xie, Kai Yang, et al. 2021. Logm: Log analysis for multiple components of hadoop platform. IEEE Access 9 (2021), 73522–73532

    2021

  46. [54]

    A. Yang, B. Xiao, B. Wang, et al. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 (2023)

    2023 arXiv

  47. [55]

    A. Yang, B. Yang, B. Hui, et al . 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671 (2024)

    2024 arXiv

  48. [56]

    Yıldız, N.K

    Ç. Yıldız, N.K. Ravichandran, P. Punia, et al . 2024. Investigating Continual Pretraining in Large Language Models: Insights and Implications. arXiv preprint arXiv:2402.17400 (2024)

    2024 arXiv

  49. [57]

    Zhang, W

    S. Zhang, W. Meng, et al. 2007. Syslog Processing for Switch Failure Diagnosis and Prediction in Datacenter Networks. In Proc. of the IEEE/ACM International Symposium on Quality of Service (IWQoS)

    2007

  50. [58]

    Zhang, S

    S. Zhang, S. Roller, N. Goyal, et al. 2022. Opt: Open Pre-Trained Transformer Language Models. arXiv preprint arXiv:2205.01068 (2022)

    2022 arXiv

  51. [59]

    Zhang, Y

    X. Zhang, Y. Xu, Q. Lin, et al. 2019. Robust Log-Based Anomaly Detection on Unstable Log Data. In Proc. of the ACM Joint Meeting on European Software Engi- neering Conference and Symposium on the Foundations of Software Engineering

    2019

  52. [60]

    Zhang, Y

    Y. Zhang, Y. Li, L. Cui, et al. 2023. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv preprint arXiv:2309.01219 (2023)

    2023 arXiv

  53. [61]

    W.X. Zhao, K. Zhou, J. Li, et al. 2023. A Survey of Large Language Models. arXiv preprint arXiv:2303.18223 (2023)

    2023 arXiv

  54. [62]

    Zheng, G

    H. Zheng, G. Chu, H. Sun, et al . 2023. LogDAPT: Log Data Anomaly Detec- tion with Domain-Adaptive Pretraining (industry track). In Proc. of the 24th International Middleware Conference: Industrial Track

    2023

  55. [63]

    Zheng, R

    Y. Zheng, R. Zhang, J. Zhang, et al. 2024. LlamaFactory: Unified Efficient Fine- Tuning of 100+ Language Models. InProc. of the Annual Meeting of the Association for Computational Linguistics. http://arxiv.org/abs/2403.13372

    2024 arXiv

  56. [64]

    J. Zhu, S. He, J. Liu, et al . 2019. Tools and Benchmarks for Automated Log Parsing. In Proc. of the IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.