Pith. sign in

REVIEW 3 major objections 5 minor 53 references

MicLog claims that progressively meta-training a small open-source LLM from zero- to few-shot examples makes it the most accurate and fastest log parser tested, reaching 97.6% parsing accuracy with a 42.4% speed-up.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:11 UTC pith:DA6DXU2H

load-bearing objection Same-source labels make the SOTA claim conditional; the progressive meta-ICL idea and ablation are still worth referee time. the 3 major comments →

arxiv 2601.07005 v1 pith:DA6DXU2H submitted 2026-01-11 cs.SE cs.AI

MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning

Jianbo Yu , Yixuan Li , Hai Xu , Kang Xu , Junjielong Xu , Zhijing Li , Pinjia He , Wanyuan Wang This is my paper
classification cs.SE cs.AI
keywords log parsingin-context learningmeta-learninglarge language modelstemplate extractionBM25progressive meta in-context learningcache
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MicLog aims to settle a practical question: can a small, locally runnable LLM match or beat much larger proprietary models at log parsing? The paper's answer is yes, provided the model is first meta-trained with a progressive in-context schedule—0-shot, 1-shot, ..., k-shot—so it learns how to exploit demonstrations rather than just repeating them. Averaged over 14 public log datasets, MicLog reports 97.6% Parsing Accuracy, 95.3% Precision Template Accuracy, and 90.5% Recall Template Accuracy, each above the prior state of the art, while a two-level cache cuts total parsing time by 42.4%. A sympathetic reader cares because this would make accurate, private, on-premises log parsing practical, and because the training schedule is a general recipe for improving small models at extraction tasks.

Core claim

The central claim is that a progressive meta in-context learning paradigm—training a 3B open-source LLM to condition on an increasing number of demonstration pairs, from 0 to 5—transforms a small model into the most accurate LLM-based log parser on the standard benchmark. MicLog achieves 97.6% Parsing Accuracy, 95.3% Precision Template Accuracy, and 90.5% Recall Template Accuracy, beating the strongest baseline by 10.3, 12.6, and 6.1 percentage points, respectively. Efficiency comes from a multi-level cache that answers repeated or structurally similar logs without a fresh LLM query, yielding a 42.4% reduction in total parsing time relative to the strongest baseline. The authors present abla

What carries the argument

ProgMeta-ICL, the progressive meta in-context learning schedule, is the load-bearing idea: during meta-training, each task is presented with 0, 1, 2, ..., k demonstration pairs and the model must predict the template for a new log, teaching it to use demonstrations of varying length. Around it, the weighted DBSCAN sampler selects diverse, representative log-template pairs from a deduplicated corpus; an enhanced BM25 retriever finds and orders the k most similar logs for the prompt; and the multi-level pre-query cache (an exact-match LRU plus a pattern cache that verifies constant segments separated by <*>) serves repeated logs without invoking the LLM.

Load-bearing premise

The headline accuracy assumes labeled log-template pairs from the same datasets being parsed are available up front to build both the meta-training set and the in-context prompts; without such same-source labels in deployment, the reported accuracy and the speed-up are not guaranteed.

What would settle it

Run MicLog on a new log source with zero labels from that source, using only other-source labels for the ICL prompts and meta-training, and measure Parsing Accuracy on a held-out sample. If it falls to the 87.3% level of the no-ProgMeta ablation or below the strongest baseline, the claim that progressive meta in-context learning transfers across log domains fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Log parsing can be run locally on commodity hardware with a 3B model, so sensitive log data never has to be sent to a proprietary API.
  • The 42.4% reduction in total parsing time makes LLM-based parsing comparable in speed to the fastest syntax-based parsers, not just a batch-only tool.
  • A single epoch of progressive meta-training is enough to beat the previous state of the art, and only about 326 labeled logs per dataset are used, so the training cost is small.
  • The pattern cache exploits temporal locality of log streams, so heavily repeated log types become nearly free to parse.
  • Wilcoxon signed-rank tests across 14 datasets give p-values below 0.05 for all three metrics, supporting that the accuracy gap is not due to a few lucky datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Deployment caveat: the paper labels the meta-training and inference support sets from the same corpus that is parsed; if an operator has no labels for a new log source, the 97.6% average should not be expected without bootstrapping, and the actual gap to baselines could shrink.
  • The Proxifier-only experiment suggests cross-system transfer, but it is one source; a leave-one-source-out study would show whether the meta-learned ICL skill generalizes across genuinely unseen log families.
  • Efficiency claims depend on cache hit rates, which are not reported per dataset; operators with highly repetitive logs will see the speed-up, while diverse single-occurrence logs will not.
  • Because BM25 selection is a major accuracy driver, embedding-based retrieval (dense retrieval) is a natural next test; the paper's own ablation implies retrieval quality, not sampling, is the second biggest lever.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MicLog, an LLM-based log parsing framework built around a 'progressive meta in-context learning' (ProgMeta-ICL) paradigm on a small open-source model (Qwen2.5-3B). Three components are introduced: a weighted DBSCAN sampler that selects diverse labeled log-template pairs from the target dataset, a progressive meta-training procedure that steps from 0-shot to k-shot ICL prompts, and a multi-level cache (LRU + pattern cache) to reduce LLM queries. On Loghub-2.0, the paper reports 97.6% Parsing Accuracy, 95.3% Precision Template Accuracy, and 90.5% Recall Template Accuracy, claimed to outperform the previous SOTA (AdaParser) by 10.3, 12.6, and 6.1 percentage points, with a 42.4% reduction in total parsing time. The evaluation compares MicLog against seven baselines across all 14 Loghub-2.0 datasets.

Significance. If the reported results survive a fair comparison, MicLog would be a meaningful advance: it is the first log parser to combine meta-learning with ICL for this task, runs on a deployable 3B open-source model, and explicitly targets cost and privacy via local inference and caching. The paper also includes useful ablations (e.g., removing ProgMeta-ICL, replacing DBSCAN, replacing BM25, and a cross-domain 'Proxifier-only' experiment) that suggest the core ideas have substance. However, the significance is conditional on resolving a fundamental evaluation-protocol issue: the headline numbers are produced with labeled examples drawn from the same target datasets that are subsequently parsed, while the baselines are not given equivalent target-domain labels. The cross-domain generalization experiment ('w/ Proxifier only') is a step in the right direction but is not the basis of the SOTA claim. The paper would be substantially stronger if the main comparison were re-run under a controlled protocol, with code and data artifacts released.

major comments (3)
  1. [Algorithm 1 (lines 2–3) and Figure 2 caption] The experimental protocol allows target-dataset labels to flow into both meta-training and ICL prompt construction. Algorithm 1 samples S_meta and S_inf from D_dedup, where D is the same dataset later parsed in Stage 3; the Figure 2 caption explicitly states these datasets 'need to be labeled before progressive meta in-context training and log parsing.' The abstract and intro further qualify the results as 'when using same-source prompt examples.' In contrast, baselines such as AdaParser, LILAC, and LibreLog are not given equivalent oracle-labeled support sets. Thus the 10.3-point PA gap over AdaParser mixes the algorithm's contribution with access to target labels. If such labels are unavailable at deployment, MicLog as specified cannot run; if they are available, the correct comparison is against a few-shot supervised baseline given the same k examples. The authors must either (a) eval
  2. [Table 2 and 'Environment and Implementation' (RQ1)] The paper states that each experimental configuration was repeated five times and means are reported, yet Table 2 gives a single number per dataset and metric with no variance, no confidence intervals, and no per-run data. The Wilcoxon signed-rank tests (p-values 0.0038, 0.0036, 0.0356) are computed over 14 datasets, but with only one reported observation per dataset it is impossible to assess whether the advantage is stable. The average gains are also not uniform: AdaParser has equal or higher RTA on OpenSSH (94.7 vs. 92.1) and higher PA on HDFS (100.0 vs. 100.0, tie) and elsewhere, so the 'highest average' claim hides per-dataset variability. Report standard deviations, per-run results, or confidence intervals, and justify the paired test with the actual repeated measurements.
  3. [Evaluation setup and implementation] The paper introduces many free hyperparameters—DBSCAN epsilon and MinPts, complexity smoothing factor factor_s, BM25 k1 and b, sampling ratio alpha, LRU capacity C_LRU and token threshold tau, max shot K, and number of epochs—but no values are given anywhere in the text or a table. Additionally, no code or data (sampled S_meta/S_inf, prompts, or split indices) is released. For an empirical SE paper, this makes the results impossible to reproduce and leaves open the possibility of hidden tuning on Loghub-2.0. The authors should release an artifact and specify all hyperparameter values; if the method is invariant to some of them, say so explicitly.
minor comments (5)
  1. [Section 'Evaluation' (RQ headings)] The research-question numbering is duplicated: RQ3 and RQ4 both appear twice ("RQ3: How efficient is MicLog?" and "RQ4: How do different training strategies affect MicLog?" after RQ1 and RQ2). Renumber consistently as RQ1–RQ4.
  2. [Equation (1)] The complexity formula is ambiguous as typeset: it reads "token_l / token_l + length_l", which could mean token_l / (token_l + length_l) or (token_l / token_l) + length_l. Use proper fraction notation and clarify units.
  3. [RQ3 efficiency text] There is a typo: "Logpub-2.0" should be "Loghub-2.0". Also, the claim that average cache time (63.5s) is substantially lower than LILAC's 376.5s is not shown in Figure 4 or any table; either include the value in the figure or give the full breakdown in a table.
  4. [Related work and references] The statement "By analyzing the work of Zhang et al. [Zhang et al. 2023]" is vague; the specific connection between the ProgMeta-ICL objective (Eq. 3) and that reference should be spelled out. Also, "paser" appears in the baselines description.
  5. [Table 3 and Table 4 formatting] The arrows and parenthetical deltas in Tables 3 and 4 are not explained in a general note; state that numbers in parentheses are changes relative to the full MicLog. Also, the "w/ k-means sampling" row reports PA 97.1 (↓0.5), but the delta vs. 97.6 is 0.5; ensure all deltas are consistent with the rounded base values.

Circularity Check

1 steps flagged

Headline accuracy is produced with same-dataset labeled prompts and no held-out split, making the SOTA comparison partially circular.

specific steps
  1. fitted input called prediction [Algorithm 1 (Stage 1 lines 2-3, Stage 3 lines 12-19); Figure 2 caption]
    "D dedup ←deduplicate(D); S meta,S inf ←WeightedDBSCAN(D dedup, α) ... foreach raw log l_i ∈ D do ... Retrieve top-k logs R ←BM25(l_i, S inf); Construct ICL prompt P← {R, l_i}, t_i ←LLM θ∗ (P) ... The ProgMeta-ICL dataset and inference dataset need to be labeled before progressive meta in-context training and log parsing."

    The ICL support set S_inf is a labeled sample of the same dataset D that the parser is then scored on. No held-out split excludes S_inf or its duplicates from the parsed logs, so the prompt for a test log l_i can contain l_i itself or near-identical labeled logs, and the multi-level cache can propagate those labeled templates to matching test logs. Accuracy is therefore measured against the same target labels that were used as inputs for meta-training and prompting; the 'prediction' is partly read off from the evaluation labels rather than being an independent generalization. The reported SOTA gap over AdaParser, whose ICL is self-generated and label-free, is not controlled for this extra access to target-domain ground truth.

full rationale

This is an empirical systems paper rather than a formal derivation, so the circularity is in the evaluation protocol rather than in equations. Algorithm 1 defines no train/inference split: S_meta and S_inf are sampled from D_dedup, and then every raw log l_i in D is parsed using BM25 retrieval from S_inf and the meta-trained LLM. Because S_inf is labeled from the same dataset being evaluated, the reported 97.6% PA / 95.3% PTA / 90.5% RTA numbers partly measure retrieval of the provided labels, especially given the cache that stores templates derived from those labeled examples. The paper is candid that the inference dataset 'need[s] to be labeled,' but the comparison with AdaParser, LILAC, and LibreLog—methods that do not receive equivalent labeled same-source support—makes the headline improvement partially circular. The 'w/ Proxifier only' ablation provides some evidence that ProgMeta-ICL itself transfers, so the result is not fully forced; the score is 6 rather than 8 or 10. There is no load-bearing self-citation chain; the main issue is same-source labeled inputs being called prediction.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The central empirical claim rests on unstated hyperparameters and on an evaluation assumption—same-source labeled support—that is disclosed but not defended. No new physical or conceptual entities are introduced; ProgMeta-ICL is a training procedure, not a postulated entity.

free parameters (7)
  • DBSCAN neighborhood radius epsilon = not reported
    Determines clustering granularity and therefore which logs are sampled for meta-training and ICL support; no value is given.
  • DBSCAN MinPts = not reported
    Minimum cluster density threshold; affects sample diversity and representativeness.
  • complexity smoothing factor_s = not reported
    Introduced in Eq. 2 to avoid numerical instability in sampling weights; chosen by hand.
  • BM25 k1 and b = not reported
    Tunable parameters in Eq. 9 controlling term-frequency saturation and length normalization; no values are stated.
  • sampling ratio alpha = 0.009% (326 logs avg.)
    Authors select this ratio for the ProgMeta-ICL and inference datasets; it affects both training quality and prompt construction.
  • LRU cache capacity C_LRU and token threshold tau = not reported
    Control cache hit rate and therefore the accuracy-efficiency trade-off.
  • max shot number K and epochs = K=5, epochs=5
    Hyperparameters of the progressive meta-training schedule; epoch=1 is also tested in the ablation.
axioms (5)
  • domain assumption Ground-truth templates in Loghub-2.0 are correct and define the parsing target.
    All metrics compare predicted templates against benchmark ground truth; any error in ground truth transfers to all methods.
  • domain assumption Labeled examples from each target dataset can be used at inference time to build ICL prompts.
    Algorithm 1 retrieves top-k examples from S_inf, a labeled subset of the same D being parsed; Figure 2 caption says the inference dataset must be labeled.
  • domain assumption Using 0.009% of each Loghub-2.0 dataset for meta-training does not substantively contaminate evaluation.
    The paper does not quantify overlap or present results on datasets fully excluded from meta-training; the Proxifier-only ablation is the only cross-distribution check.
  • domain assumption BM25-ranked examples are the most useful demonstrations for the LLM.
    The method section cites prior work that example selection matters, but no evidence is given here that BM25 ranking is optimal for log parsing.
  • domain assumption Qwen2.5-3B's in-context learning ability improves with gradient meta-training on log parsing tasks.
    The ProgMeta-ICL objective in Eqs. 3-4 assumes a task-agnostic shared parameterization can be learned from a small sample; no theoretical guarantee is offered.

pith-pipeline@v1.3.0-alltime-deepseek · 14500 in / 13476 out tokens · 138560 ms · 2026-08-03T11:11:59.755066+00:00 · methodology

0 comments
read the original abstract

Log parsing converts semi-structured logs into structured templates, forming a critical foundation for downstream analysis. Traditional syntax and semantic-based parsers often struggle with semantic variations in evolving logs and data scarcity stemming from their limited domain coverage. Recent large language model (LLM)-based parsers leverage in-context learning (ICL) to extract semantics from examples, demonstrating superior accuracy. However, LLM-based parsers face two main challenges: 1) underutilization of ICL capabilities, particularly in dynamic example selection and cross-domain generalization, leading to inconsistent performance; 2) time-consuming and costly LLM querying. To address these challenges, we present MicLog, the first progressive meta in-context learning (ProgMeta-ICL) log parsing framework that combines meta-learning with ICL on small open-source LLMs (i.e., Qwen-2.5-3B). Specifically, MicLog: i) enhances LLMs' ICL capability through a zero-shot to k-shot ProgMeta-ICL paradigm, employing weighted DBSCAN candidate sampling and enhanced BM25 demonstration selection; ii) accelerates parsing via a multi-level pre-query cache that dynamically matches and refines recently parsed templates. Evaluated on Loghub-2.0, MicLog achieves 10.3% higher parsing accuracy than the state-of-the-art parser while reducing parsing time by 42.4%.

Figures

Figures reproduced from arXiv: 2601.07005 by Hai Xu, Jianbo Yu, Junjielong Xu, Kang Xu, Pinjia He, Wanyuan Wang, Yixuan Li, Zhijing Li.

Figure 1
Figure 1. Figure 1: A simple process of Log Parsing. To address these limitations, we propose MicLog, an effective and efficient progressive meta in-context learn￾ing (ProgMeta-ICL) log parsing framework that leverages ProgMeta-ICL to enhance the ICL capabilities of LLMs in log parsing tasks. Specifically, MicLog comprises three main components: a weighted DBSCAN sampler, a pro￾gressive meta in-context training module, and a … view at source ↗
Figure 2
Figure 2. Figure 2: The workflow of MicLog framework. Sampling and ProgMeta-ICL are performed before online log parsing. The [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Robustness comparison between baselines and MicLog on public datasets (%) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Efficiency of MicLog and baselines on Loghub-2.0 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 18 linked inside Pith

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Aky \"u rek, E.; Schuurmans, D.; Andreas, J.; Ma, T.; and Zhou, D. 2022. What learning algorithm is in-context learning? investigations with linear models. arXiv preprint arXiv:2211.15661

    Pith/arXiv arXiv 2022

  4. [4]

    Amar, A.; and Rigby, P. C. 2019. Mining historical test logs to predict bugs and localize faults in the test logs. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), 140--151. IEEE

    2019

  5. [5]

    A.; Soykan, E

    Bilgin, Z.; Ersoy, M. A.; Soykan, E. U.; Tomur, E.; C omak, P.; and Kara c ay, L. 2020. Vulnerability prediction from source code using machine learning. IEEE Access, 8: 150672--150684

    2020

  6. [6]

    Brown, T. B. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

    Pith/arXiv arXiv 2020

  7. [7]

    Chen, Y.; Zhong, R.; Zha, S.; Karypis, G.; and He, H. 2021. Meta-learning via language model in-context tuning. arXiv preprint arXiv:2110.07814

    Pith/arXiv arXiv 2021

  8. [8]

    Coda-Forno, J.; Binz, M.; Akata, Z.; Botvinick, M.; Wang, J.; and Schulz, E. 2023. Meta-in-context learning in large language models. Advances in Neural Information Processing Systems, 36: 65189--65201

    2023

  9. [9]

    Dai, H.; Li, H.; Chen, C.-S.; Shang, W.; and Chen, T.-H. 2020. Logram: Efficient log parsing using n n-gram dictionaries. IEEE Transactions on Software Engineering, 48(3): 879--892

    2020

  10. [10]

    Dong, Q.; Li, L.; Dai, D.; Zheng, C.; Ma, J.; Li, R.; Xia, H.; Xu, J.; Wu, Z.; Liu, T.; et al. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234

    Pith/arXiv arXiv 2022

  11. [11]

    Du, M.; and Li, F. 2016. Spell: Streaming parsing of system event logs. In 2016 IEEE 16th International Conference on Data Mining (ICDM), 859--864. IEEE

    2016

  12. [12]

    Du, M.; Li, F.; Zheng, G.; and Srikumar, V. 2017. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, 1285--1298

    2017

  13. [13]

    Ester, M.; Kriegel, H.-P.; Sander, J.; Xu, X.; et al. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, 226--231

    1996

  14. [14]

    Han, Z.; Li, X.; Xing, Z.; Liu, H.; and Feng, Z. 2017. Learning to predict severity of software vulnerability using only vulnerability description. In 2017 IEEE International conference on software maintenance and evolution (ICSME), 125--136. IEEE

    2017

  15. [15]

    He, P.; Zhu, J.; Zheng, Z.; and Lyu, M. R. 2017. Drain: An online log parsing approach with fixed depth tree. In 2017 IEEE international conference on web services (ICWS), 33--40. IEEE

    2017

  16. [16]

    He, S.; He, P.; Chen, Z.; Yang, T.; Su, Y.; and Lyu, M. R. 2021. A survey on automated log analysis for reliability engineering. ACM computing surveys (CSUR), 54(6): 1--37

    2021

  17. [17]

    R.; and Zhang, D

    He, S.; Lin, Q.; Lou, J.-G.; Zhang, H.; Lyu, M. R.; and Zhang, D. 2018. Identifying impactful service system problems via log analysis. In Proceedings of the 2018 26th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, 60--70

    2018

  18. [18]

    He, S.; Zhu, J.; He, P.; and Lyu, M. R. 2016. Experience report: System log analysis for anomaly detection. In 2016 IEEE 27th international symposium on software reliability engineering (ISSRE), 207--218. IEEE

    2016

  19. [19]

    Huang, J.; Jiang, Z.; Chen, Z.; and Lyu, M. R. 2024 a . LUNAR: Unsupervised LLM-based log parsing. arXiv preprint arXiv:2406.07174

    Pith/arXiv arXiv 2024

  20. [20]

    Huang, J.; Jiang, Z.; Liu, J.; Huo, Y.; Gu, J.; Chen, Z.; Feng, C.; Dong, H.; Yang, Z.; and Lyu, M. R. 2024 b . Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE), 511--522. IEEE

    2024

  21. [21]

    Huo, Y.; Su, Y.; Lee, C.; and Lyu, M. R. 2023. Semparser: A semantic parser for log analytics. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 881--893. IEEE

    2023

  22. [22]

    Jiang, Z.; Liu, J.; Chen, Z.; Li, Y.; Huang, J.; Huo, Y.; He, P.; Gu, J.; and Lyu, M. R. 2024 a . LILAC: Log parsing using LLMs with adaptive parsing cache. Proceedings of the ACM on Software Engineering, 1(FSE): 137--160

    2024

  23. [23]

    Jiang, Z.; Liu, J.; Huang, J.; Li, Y.; Huo, Y.; Gu, J.; Chen, Z.; Zhu, J.; and Lyu, M. R. 2024 b . A large-scale evaluation for log parsing techniques: How far are we? In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, 223--234

    2024

  24. [24]

    A.; Shin, D.; Bianculli, D.; and Briand, L

    Khan, Z. A.; Shin, D.; Bianculli, D.; and Briand, L. 2022. Guidelines for assessing the accuracy of log message template identification techniques. In Proceedings of the 44th International Conference on Software Engineering, 1095--1106

    2022

  25. [25]

    Le, V.-H.; and Zhang, H. 2023. Log parsing with prompt-based few-shot learning. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2438--2449. IEEE

    2023

  26. [26]

    Li, G.; Wang, P.; and Ke, W. 2023. Revisiting large language models as zero-shot relation extractors. arXiv preprint arXiv:2310.05028

    Pith/arXiv arXiv 2023

  27. [27]

    Li, G.; Wang, P.; Liu, J.; Guo, Y.; Ji, K.; Shang, Z.; and Xu, Z. 2024 a . Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors. arXiv preprint arXiv:2404.17807

    Pith/arXiv arXiv 2024

  28. [28]

    Li, Z.; Fu, Q.; Huang, Z.; Yu, J.; Li, Y.; Lai, Y.; and Ma, Y. 2024 b . Revisiting Log Parsing: The Present, the Future, and the Uncertainties. IEEE Transactions on Reliability

    2024

  29. [29]

    Lin, Q.; Zhang, H.; Lou, J.-G.; Zhang, Y.; and Chen, X. 2016. Log clustering based problem identification for online service systems. In Proceedings of the 38th International Conference on Software Engineering Companion, 102--111

    2016

  30. [30]

    Liu, Y. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    Pith/arXiv arXiv 2019

  31. [31]

    Liu, Y.; Zhang, X.; He, S.; Zhang, H.; Li, L.; Kang, Y.; Xu, Y.; Ma, M.; Lin, Q.; Dang, Y.; et al. 2022. Uniparser: A unified log parser for heterogeneous log data. In Proceedings of the ACM Web Conference 2022, 1893--1901

    2022

  32. [32]

    Ma, R.; Zhou, X.; Gui, T.; Tan, Y.; Li, L.; Zhang, Q.; and Huang, X. 2021. Template-free prompt tuning for few-shot NER. arXiv preprint arXiv:2109.13532

    Pith/arXiv arXiv 2021

  33. [33]

    J.; and Chen, T.-H

    Ma, Z.; Kim, D. J.; and Chen, T.-H. 2024. LibreLog: Accurate and Efficient Unsupervised Log Parsing Using Open-Source Large Language Models. arXiv preprint arXiv:2408.01585

    Pith/arXiv arXiv 2024

  34. [34]

    Min, S.; Lewis, M.; Zettlemoyer, L.; and Hajishirzi, H. 2021. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943

    Pith/arXiv arXiv 2021

  35. [35]

    Min, S.; Lyu, X.; Holtzman, A.; Artetxe, M.; Lewis, M.; Hajishirzi, H.; and Zettlemoyer, L. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837

    Pith/arXiv arXiv 2022

  36. [36]

    Rey, D.; and Neuh \"a user, M. 2011. Wilcoxon-signed-rank test. In International encyclopedia of statistical science, 1658--1659. Springer

    2011

  37. [37]

    Robertson, S. 2004. Understanding inverse document frequency: on theoretical arguments for IDF. Journal of documentation, 60(5): 503--520

    2004

  38. [38]

    Robertson, S.; Zaragoza, H.; et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval , 3(4): 333--389

    2009

  39. [39]

    Rubin, O.; Herzig, J.; and Berant, J. 2021. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633

    Pith/arXiv arXiv 2021

  40. [40]

    P.; and Xu, X

    Schubert, E.; Sander, J.; Ester, M.; Kriegel, H. P.; and Xu, X. 2017. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Transactions on Database Systems (TODS), 42(3): 1--21

    2017

  41. [41]

    Shima, K. 2016. Length matters: Clustering system log messages using length of words. arXiv preprint arXiv:1611.03213

    Pith/arXiv arXiv 2016

  42. [42]

    Wu, Y.; Yu, S.; and Li, Y. 2024. Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction. arXiv preprint arXiv:2406.03376

    Pith/arXiv arXiv 2024

  43. [43]

    Xiao, Y.; Le, V.-H.; and Zhang, H. 2024. Free: Towards More Practical Log Parsing with Large Language Models. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, 153--165

    2024

  44. [44]

    Xu, J.; Cui, Z.; Zhao, Y.; Zhang, X.; He, S.; He, P.; Li, L.; Kang, Y.; Lin, Q.; Dang, Y.; et al. 2024 a . Unilog: Automatic logging via llm and in-context learning. In Proceedings of the 46th ieee/acm international conference on software engineering, 1--12

    2024

  45. [45]

    Xu, J.; Yang, R.; Huo, Y.; Zhang, C.; and He, P. 2024 b . DivLog: Log Parsing with Prompt Enhanced In-Context Learning. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, 1--12

    2024

  46. [46]

    Yang, A.; Yang, B.; Hui, B.; Zheng, B.; Yu, B.; Zhou, C.; Li, C.; Li, C.; Liu, D.; Huang, F.; Dong, G.; Wei, H.; Lin, H.; Tang, J.; Wang, J.; Yang, J.; Tu, J.; Zhang, J.; Ma, J.; Xu, J.; Zhou, J.; Bai, J.; He, J.; Lin, J.; Dang, K.; Lu, K.; Chen, K.; Yang, K.; Li, M.; Xue, M.; Ni, N.; Zhang, P.; Wang, P.; Peng, R.; Men, R.; Gao, R.; Lin, R.; Wang, S.; Bai...

    Pith/arXiv arXiv 2024

  47. [47]

    Yao, Y.; Duan, J.; Xu, K.; Cai, Y.; Sun, Z.; and Zhang, Y. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, 100211

    2024

  48. [48]

    Yu, S.; He, P.; Chen, N.; and Wu, Y. 2023. Brain: Log parsing with bidirectional parallel tree. IEEE Transactions on Services Computing, 16(5): 3224--3237

    2023

  49. [49]

    Zhang, X.; Xu, Y.; Lin, Q.; Qiao, B.; Zhang, H.; Dang, Y.; Xie, C.; Yang, X.; Cheng, Q.; Li, Z.; et al. 2019. Robust log-based anomaly detection on unstable log data. In Proceedings of the 2019 27th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, 807--817

    2019

  50. [50]

    Zhang, Y.; Zhang, F.; Yang, Z.; and Wang, Z. 2023. What and how does in-context learning learn? bayesian model averaging, parameterization, and generalization. arXiv preprint arXiv:2305.19420

    Pith/arXiv arXiv 2023

  51. [51]

    X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al

    Zhao, W. X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2)

    Pith/arXiv arXiv 2023

  52. [52]

    Zhong, A.; Mo, D.; Liu, G.; Liu, J.; Lu, Q.; Zhou, Q.; Wu, J.; Li, Q.; and Wen, Q. 2024. LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4559--4570

    2024

  53. [53]

    Zhu, J.; He, S.; Liu, J.; He, P.; Xie, Q.; Zheng, Z.; and Lyu, M. R. 2019. Tools and benchmarks for automated log parsing. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 121--130. IEEE

    2019