Pith. sign in

REVIEW 3 major objections 6 minor 85 references

LLMLog: Advanced Log Template Generation via LLM-driven Multi-Round Annotation

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Log-template AI hits perfect scores on 11 of 16 datasets

desk verdict Solid empirical system with a broken theoretical guarantee as stated; worth reviewing but needs a fix. read the letter →

arxiv 2508.09594 v1 pith:XSTO5QPW submitted 2025-08-13 cs.DB

Fei Teng , Haoyang Li , Lei Chen This is my paper
classification cs.DB
keywords logtemplategenerationLLMin-contextlearningmulti-roundannotationsemanticeditdistanceadaptivedemonstrationselectionparsingactivecost-efficientprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that log template generation—turning raw system logs like "2024-11-14 192.168.1.1 GET /index.html 200" into structured templates like "[DATE] [IP] [RESOURCE] [STATUS]"—can be automated more accurately and cheaply than current heuristic, neural, and LLM baselines by letting the LLM choose its own examples. The proposed framework, LLMLog, runs in rounds: it measures log similarity with a semantic edit distance that emphasizes keywords rather than long timestamps, it spends each round's human-annotation budget on logs that are both representative and hard for the LLM, and it feeds each unlabeled log a minimal set of fully labeled demonstration logs that cover every word in the input. Experiments on 16 LogPAI datasets report LLMLog reaching or beating all baselines on message-level and template-level accuracy, often at 100 percent, while using fewer tokens and lower API cost. If the claim holds, it means accurate templates can be produced with much less human labeling and cheaper LLM prompting than the previous state of the art.

What carries the argument

Semantic Edit Distance (SED): a dynamic-programming edit distance on the remaining words of two logs (after removing words already recognized by the labeled set), with replacement cost 0 when two word embeddings have cosine similarity $\geq 0$ and 1 otherwise. It carries the argument because it defines both the representative score (which logs cover many peers) and the coverage objective for demonstration selection; two submodular greedy algorithms with approximation guarantees run on top of it.

What would settle it

Construct or find a set of logs where every word of an unlabeled log has cosine similarity $\geq 0$ to words in a labeled log that actually carries a different template label; if the LLM then copies the wrong template instead of the correct one, the coverage condition is not sufficient to guarantee accuracy.

Watch

Extended reading notes

Core claim

The paper claims that log template generation is best done by an iterative loop in which (1) unlabeled logs are compared by a semantic edit distance (SED) that removes words already identified by the current labeled set, (2) each round's annotation budget is spent on logs that are simultaneously representative and hard for the LLM, and (3) each input log receives a minimal set of demonstrations that covers all of its words with embeddings having non-negative cosine similarity. On the 16 LogPAI datasets, this design is reported to yield higher MLA, PTA, and RTA than DivLog, AdaICL, Drain, and LogPPT on every dataset, reaching 100% on a majority of them, while using fewer tokens and lower API

Load-bearing premise

The method's gains rest on the assumption that any two words with non-negative embedding cosine similarity carry the same template label, so covering each input word with such a labeled word transfers the correct label; if that transfer fails, the selected demonstrations and annotations need not help.

Editorial extensions

If this is right

  • Log template generation can run with far fewer human annotations because each round targets the most representative and most uncertain logs.
  • Accuracy becomes less dependent on the choice of $k$ in $k$-shot prompting and on embedding-based similarity, since demonstrations are chosen adaptively per log.
  • The same framework lowers API cost and inference time because each unlabeled log gets only as many demonstrations as needed to cover its words.
  • The performance ceiling is tied to the annotation budget; accuracy stabilizes once the budget covers the distinct words, making tuning predictable.
  • The reported gains appear on both GPT-4o and a 7B open model, suggesting the mechanism is not tied to one LLM's strength.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The coverage condition $\cos(w_i, w_j) \geq 0$ is very permissive for typical embedding spaces, so the real driver of the gains may be the greedy minimum-covering selection rather than the semantic threshold; a control experiment using random word embeddings would separate the two.
  • If the mechanism holds, adaptive demonstration selection should transfer to other token-level sequence annotation tasks (e.g., column type annotation, NER-style slot filling), where fixed $k$-shot contexts are also standard.
  • The adaptive budget rule (shrink budget as identified words grow) creates a self-referential loop: small early rounds may under-cover words and inflate later budgets, so stability depends on the hand-set first two rounds' budgets more than the paper's sensitivity analysis shows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. LLMLog proposes an LLM-driven multi-round annotation framework for log template generation. It introduces a semantic edit-distance (SED) similarity, a representative score, an LLM prediction confidence score, a greedy annotation selection with claimed approximation guarantees, an adaptive demonstration-selection algorithm based on word coverage, and an adaptive per-round budget. The paper reports extensive experiments on 16 datasets with GPT-4o and Qwen2.5-7B-Instruct, showing that LLMLog matches or beats DivLog, AdaICL, Drain, and LogPPT on MLA, PTA, and RTA while reducing inference time and API cost. The main formal claims are a 1-1/e approximation for the annotation selection and a 1+ln(n) approximation for adaptive demonstration selection.

Significance. If the formal results are repaired, this would be a solid empirical contribution: the evaluation spans 16 datasets, two LLM backbones, three accuracy metrics, and cost/time measures, and a public artifact link is provided. The ablation study isolates the main components and shows large drops when SED, representativeness, confidence, adaptive demonstrations, or adaptive budget are removed. I also agree with the reader's circularity assessment: selection uses LLM probabilities and word consistency, not benchmark labels, so there is no obvious feedback loop of fitted constants. However, the theoretical guarantee for the annotation algorithm is not established as written: the objective in Eq. (8) and the proof in Appendix 6.2 are different set functions, and the proof itself omits a term in the marginal. One pseudocode loop is also non-terminating as written. These issues are load-bearing for the paper's stated contributions, though they appear fixable without changing the empirical core.

major comments (3)
  1. [3.3.3, Eq. (8); Appendix 6.2] The formal guarantee of Algorithm 1 is not established. Equation (8) defines IS(L_r)=max \sum_{s_i\in L_r} [(1-\lambda)|\cup_{i=0}^{|L_r|} I_{s_i}|/|U| + \lambda C(s_i,\hat t_i,\hat s_i)], i.e., the coverage ratio sits inside the summation. Under this objective the marginal gain of adding s is \lambda C(s)+(1-\lambda)((|L|+1)\cdot cov(L\cup{s}) - |L|\cdot cov(L))/|U|, not the appendix's \lambda C(s)+(1-\lambda)\Delta cov/|U|. The function |L|\cdot cov(L) is not submodular, so the claimed 1-1/e ratio in Theorem 2 does not follow. Appendix 6.2 analyzes a different objective, (1-\lambda)cov(L)/|U| + \lambda\sum C(s_i); moreover Eqs. (15)-(16) omit the -cov(L'_r) term in the marginal, so the submodularity proof is invalid for that objective too. Since Algorithm 1 line 8 states it computes Eq. (8), the approximation guarantee as stated is unsupported.
  2. [3.3.5, Algorithm 2] Algorithm 2's loop condition is 'while B >= 0'. Whenever the remaining budget B is smaller than the computed B_r, line 11 sets B_r = min(B_r,B)=B and then B = B - B_r = 0, but the loop does not exit. At the next iteration B_r = min(B_r,0)=0 and B remains 0, so the loop never terminates (and would repeatedly call AnnotationSelection with budget 0). The condition should be 'while B > 0', with an explicit break if B_r = 0. As written, the multi-round procedure is not a terminating algorithm.
  3. [3.4, Eq. (14); Figure 4] The word-coverage condition in Problem 1 and Algorithm 3, cosine(w_i, w_k) >= 0, is extremely permissive. With typical text embeddings, most (even unrelated) word pairs have cosine similarity >= 0; then any labeled log can 'cover' almost any word, and the set-cover constraint is vacuous. The parameter-sensitivity experiment in Figure 4 reports no change over thresholds 0, 0.25, 0.5, 0.75, 1, which is more consistent with an inactive condition than with robustness. Please report the distribution of pairwise cosine similarities between words from different templates and the number of demonstrations selected as a function of the threshold. Without this, the mechanism by which adaptive demonstration selection improves accuracy is not established. The same threshold also appears in the SED definition, Eq. (3).
minor comments (6)
  1. [3.2 / 3.3.3] Definition 1 appears twice: once for the Log-Word Bipartite Graph in Section 3.2 and again for the LLM-driven Log Annotation Problem in Section 3.3.3. Renumber one of them.
  2. [3.3.4, Theorem 1 proof sketch] The proof sketch says 'We prove Theorem 3 by reduction from the Max Coverage problem', but it should refer to Theorem 1.
  3. [Eq. (8)] The union \cup_{i=0}^{|L_r|} I_{s_i} is written inside the summation over s_i. If a global coverage term is intended, it should be placed outside the sum, and the index/domain should be clarified.
  4. [Eq. (3)] The sentence defining c(w1,w2) has a typo: 'if the word similarity between w1 and is greater than 0' is missing the second argument w2.
  5. [Theorem 4 / Appendix 6.4] The approximation ratio 1+ln(n) needs n defined. Also, the standard set-cover bound is H(max set size) <= 1+ln n; please cite the precise form being used.
  6. [Figure 4] The text says the effect of the cosine threshold 'has converged'. Please quantify what is shown: are all curves identical, or within some tolerance? Reporting the actual values would make the claim checkable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the LLM-confidence-driven annotation loop is active learning, not a circular derivation.

full rationale

LLMLog is an active-learning pipeline: it selects logs for human annotation using (a) an SED-based representativeness score defined from an edit-distance over word embeddings and (b) an LLM self-confidence score based on the model's own predicted probabilities and word-consistency. Neither component uses ground-truth template labels during selection. The final predictions on unlabeled logs are generated by the same LLM after augmenting demonstrations, but the selection signal is not the target label; it is a model-internal estimate. This is an active-learning loop, not a case where a fitted parameter is renamed as a prediction. The SED metric, the coverage constraint (Problem 1, Eq. 14), and the greedy objectives (Eqs. 8 and 13) are all defined independently of the benchmark labels and of the final accuracy metrics. Self-citations to the technical report [65] are pointers to proofs that appear in the paper's own appendix; they are not load-bearing as independent evidence. The empirical comparisons in Tables 3 and 6 are against external baselines and do not reduce to the paper's own fitted values. A separate formal concern exists, but it is not circularity: the proof in Appendix 6.2 (Lemma 1) analyzes a marginal of the form C(s_i,...) + |union(I_l) ∪ I_{s_i}|/|U|, while Eq. (8) writes the coverage term inside the summation over s_i, so the written 1-1/e guarantee may not follow from the stated objective. That is a proof-correctness issue, not a derivation that assumes its own conclusion. Overall, no circular step meeting the evidentiary bar is present.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on domain assumptions about word similarity and LLM confidence, plus hyperparameters including lambda, delta, alpha, and annotation budgets. No new physical or conceptual entities are introduced; SED and confidence scores are mathematical tools.

free parameters (6)
  • lambda = 0.5
    Trade-off between representative coverage and LLM confidence in Eq. (8); set to 0.5 for all datasets after sensitivity analysis on Hadoop/Proxifier (Section 4.4.2).
  • delta = 0.5
    Threshold parameter in Eq. (4) defining representative sets; set to 0.5 for all datasets after sensitivity analysis (Section 4.4.3).
  • alpha
    Weight in Eq. (7) between average token probability and word consistency; default not reported in the paper, only varied in the case study, so exact replication requires the code.
  • total annotation budget B = 50 or 200 per dataset
    Annotation budget is set per dataset based on template and word counts (Section 4.1.4), affecting the accuracy results.
  • initial round budgets B0, B1 = 10/10 for B=50; 50/25 for B=200
    Manual startup budgets for the adaptive budget schedule in Eq. (11); required to reproduce the multi-round behavior.
  • cosine similarity threshold = 0
    Threshold in Eq. (3) and Eq. (14) determining word similarity; set to 0 for all runs and tested in Figure 4.
assumptions (5)
  • domain assumption Logs that share the same template share important keywords, and edit-distance over words therefore reflects template similarity.
    Central to SED and the representative score (Section 3.2, Eq. (2), Section 3.3.1); if templates do not share surface words, SED-based selection loses its meaning.
  • domain assumption Word embedding cosine similarity >= 0 is a valid proxy for 'same template token'.
    Used in Eq. (3) for replacement cost and in Eq. (14) to define coverage; no evidence in the paper that threshold 0 corresponds to label-transferable similarity.
  • domain assumption The LLM's average token probability and word-consistency indicator are meaningful signals of which logs benefit from annotation.
    Confidence score Eq. (7) drives annotation selection; the paper cites hallucination but does not validate the score against ground truth.
  • domain assumption Ground-truth templates in the 16 Log-PAI datasets are correct and the 2,000-log samples are representative.
    All accuracy numbers depend on these labels (Section 4.1.1).
  • standard math NP-hardness and approximation results for Max Coverage and Set Cover transfer to the constructed instances.
    Used in Theorems 1 to 4 and Appendix 6.1 to 6.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLMLog: Advanced Log Template Generation via LLM-driven Multi-Round Annotation." pith.science (2026). https://pith.science/paper/XSTO5QPW

@misc{pith2026250809594,
  author       = {Pith},
  title        = {Pith review of: LLMLog: Advanced Log Template Generation via LLM-driven Multi-Round Annotation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XSTO5QPW}},
  note         = {Machine review of arXiv:2508.09594}
}
abstract

Modern computing systems, such as HDFS and Spark, produce vast quantities of logs that developers use for tasks like anomaly detection and error analysis. To simplify log analysis, template generation methods have been proposed to standardize log formats, transforming unstructured data into structured templates. Existing heuristic-based methods and neural network-based methods suffer from low accuracy problems due to the reliance on handcrafted heuristics or specific log patterns in training sets. Recently, large language models (LLMs) have shown great potential in log template generation. However, they often struggle with ambiguous, complex, or highly specific log content, which can lead to errors in generating accurate templates. To address these challenges, we propose LLMLog, a multi-round annotation framework with adaptive in-context learning. We first propose an edit-distance-based similarity metric to evaluate log similarity. Then, we introduce a method to select the most informative $k$ unlabeled logs for annotation by considering both the representativeness of the logs and the confidence of LLM predictions. Additionally, we design an adaptive context selection strategy that adaptively selects labeled logs to ensure comprehensive keyword coverage for unlabeled logs. These labeled logs serve as the context for LLMs to better understand the unlabeled logs, thereby enhancing the accuracy of template generation. Extensive experiments on sixteen datasets demonstrate that LLMLog outperforms the state-of-the-art approaches.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

85 extracted references · 39 canonical work pages

  1. [1]

    Anunay Amar and Peter C. Rigby. 2019. Mining historical test logs to predict bugs and localize faults in the test logs. In Proceedings of the 41st International Conference on Software Engineering (Montreal, Quebec, Canada) (ICSE ’19). IEEE Press, 140–151. https://doi.org/10.1109/ICSE.2019.00031

    arXiv 2019

  2. [2]

    Apache Software Foundation. [n.d.]. Hadoop. https://hadoop.apache.org

  3. [4]

    Bagban and Prakash J

    Tanveer I. Bagban and Prakash J. Kulkarni. 2020. Template Based Clustering of Web Documents Using Locality Sensitive Hashing (LSH). In Computing in Engineering and Technology , Brijesh Iyer, P. S. Deshpande, S. C. Sharma, and Ulhas Shiurkar (Eds.). Springer Singapore, Singapore, 567–584

    2020

  4. [5]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders. In First Conference on Language Modeling . https://openreview.net/forum?id=IW1PR7vEBf

    2024

  5. [6]

    Dragut, and Weiyi Meng

    Satadisha Saha Bhowmick, Eduard C. Dragut, and Weiyi Meng. 2023. Globally Aware Contextual Embeddings for Named Entity Recognition in Social Media Streams. In 2023 IEEE 39th International Conference on Data Engineering (ICDE) . 1544–1557. https://doi.org/10.1109/ICDE55515.2023.00122

    arXiv 2023

  6. [7]

    Angela Bonifati, Wim Martens, and Thomas Timm. 2017. An analytical study of large SPARQL query logs. Proc. VLDB Endow. 11, 2 (Oct. 2017), 149–161. https://doi.org/10.14778/3149193.3149196

    work page arXiv 2017

  7. [8]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  8. [9]

    Kayed, M.R

    Chia-Hui Chang, M. Kayed, M.R. Girgis, and K.F. Shaalan. 2006. A Survey of Web Information Extraction Systems. IEEE Transactions on Knowledge and Data Engineering 18, 10 (2006), 1411–1428. https://doi.org/10.1109/TKDE.2006.152

    work page doi:10.1109/tkde.2006.152 2006

Show all 85 references
  1. [10]

    Laming Chen, Guoxin Zhang, and Hanning Zhou. 2018. Fast greedy MAP infer- ence for determinantal point process to improve recommendation diversity. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montréal, Canada) (NIPS’18). Curran...

    2018

  2. [12]

    Robert Christensen and Feifei Li. 2013. Adaptive log compression for massive log data. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (New York, New York, USA) (SIGMOD ’13). Association for Computing Machinery, New York, NY, USA, 1283–1284...

    2013

  3. [13]

    V. Chvatal. 1979. A Greedy Heuristic for the Set-Covering Problem. Math. Oper. Res. 4, 3 (Aug. 1979), 233–235. https://doi.org/10.1287/moor.4.3.233

    1979 doi

  4. [14]

    Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. 2017. DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning. In Pro- ceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). Association f...

    2017

  5. [15]

    Ju Fan, Guoliang Li, and Lizhu Zhou. 2011. Interactive SQL query suggestion: Making databases user-friendly. In 2011 IEEE 27th International Conference on Data Engineering. 351–362. https://doi.org/10.1109/ICDE.2011.5767843

    2011

  6. [18]

    Elmore, Michael J

    Raul Castro Fernandez, Aaron J. Elmore, Michael J. Franklin, Sanjay Krishnan, and Chenhao Tan. 2023. How Large Language Models Will Disrupt Data Management. Proc. VLDB Endow. 16, 11 (July 2023), 3302–3309. https://doi.org/10.14778/ 3611479.3611527

    2023

  7. [19]

    Benjamin Feuer, Yurong Liu, Chinmay Hegde, and Juliana Freire. 2024. ArcheType: A Novel Framework for Open-Source Column Type Annotation Using Large Language Models. Proc. VLDB Endow. 17, 9 (Aug. 2024), 2279–2292. https://doi.org/10.14778/3665844.3665857

    2024

  8. [20]

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2024. Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation. Proc. VLDB Endow. 17, 5 (May 2024), 1132–1145. https://doi.org/10.14778/3641204.3641221

    2024

  9. [21]

    Daniel Golovin and Andreas Krause. 2011. Adaptive submodularity: theory and applications in active learning and stochastic optimization. J. Artif. Int. Res. 42, 1 (Sept. 2011), 427–486

    2011

  10. [22]

    Hossein Hamooni, Biplob Debnath, Jianwu Xu, Hui Zhang, Guofei Jiang, and Abdullah Mueen. 2016. LogMine: Fast Pattern Recognition for Log Analytics. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (Indianapolis, Indiana, USA) (...

    2016

  11. [23]

    Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. 2017. Drain: An Online Log Parsing Approach with Fixed Depth Tree. In 2017 IEEE International Conference on Web Services (ICWS). 33–40. https://doi.org/10.1109/ICWS.2017.13

    2017 doi

  12. [24]

    Lyu, and Dongmei Zhang

    Shilin He, Qingwei Lin, Jian-Guang Lou, Hongyu Zhang, Michael R. Lyu, and Dongmei Zhang. 2018. Identifying impactful service system problems via log analysis. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Fou...

    2018

  13. [25]

    Shilin He, Jieming Zhu, Pinjia He, and Michael R. Lyu. 2016. Experience Report: System Log Analysis for Anomaly Detection. In 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) . 207–218. https://doi.org/ 10.1109/ISSRE.2016.21

    2016 doi

  14. [27]

    Peng Jia, Pinghui Wang, Junzhou Zhao, Ye Yuan, Jing Tao, and Xiaohong Guan

  15. [28]

    Wenqi Jiang, Marco Zeller, Roger Waleffe, Torsten Hoefler, and Gustavo Alonso

  16. [29]

    Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis, and Anas- tasia Ailamaki. 2010. Aether: a scalable approach to logging. Proc. VLDB Endow. 3, 1–2 (Sept. 2010), 681–692. https://doi.org/10.14778/1920841.1920928

    2010

  17. [30]

    Hung-Yu Kao, Shian-Hua Lin, Jan-Ming Ho, and Ming-Syan Chen. 2004. Min- ing Web informative structures and contents based on entropy analysis. IEEE Transactions on Knowledge and Data Engineering 16, 1 (2004), 41–55. https: //doi.org/10.1109/TKDE.2004.1264821

    2004 arXiv

  18. [31]

    Zanis Ali Khan, Donghwan Shin, Domenico Bianculli, and Lionel Briand. 2022. Guidelines for assessing the accuracy of log message template identification tech- niques. In Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE ’...

    2022

  19. [32]

    Samir Khuller, Anna Moss, and Joseph (Seffi) Naor. 1999. The budgeted maximum coverage problem. Inf. Process. Lett. 70, 1 (April 1999), 39–45. https://doi.org/10. 1016/S0020-0190(99)00031-9

    1999

  20. [33]

    Van-Hoang Le and Hongyu Zhang. 2023. Log Parsing with Prompt-Based Few- Shot Learning. In Proceedings of the 45th International Conference on Software Engineering (Melbourne, Victoria, Australia) (ICSE ’23). IEEE Press, 2438–2449. https://doi.org/10.1109/ICSE48619.2023.00204

    2023

  21. [34]

    Guoliang Li, Xuanhe Zhou, and Xinyang Zhao. 2024. LLM for Data Management. Proceedings of the VLDB Endowment 17, 12 (2024), 4213–4216

    2024

  22. [35]

    Haoyang Li, Yiming Li, Anxin Tian, Tianhao Tang, Zhanchao Xu, Xuejia Chen, Nicole Hu, Wei Dong, Qing Li, and Lei Chen. 2024. A survey on large lan- guage model acceleration based on kv cache management. arXiv preprint arXiv:2412.19442 (2024)

    2024 arXiv

  23. [36]

    Xiaoyun Li, Pengfei Chen, Linxiao Jing, Zilong He, and Guangba Yu. 2020. Swiss- Log: Robust and Unified Deep Learning Based Log Anomaly Detection for Diverse Faults. 92–103. https://doi.org/10.1109/ISSRE5003.2020.00018

    2020

  24. [37]

    Zhaodonghui Li, Haitao Yuan, Huiming Wang, Gao Cong, and Lidong Bing

  25. [38]

    Qingwei Lin, Hongyu Zhang, Jian-Guang Lou, Yu Zhang, and Xuewei Chen

  26. [39]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55, 9, Article 195 (Jan. 2023), 35 pages. https://doi.org/1...

    2023 doi

  27. [40]

    LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency. Proc. VLDB Endow. 18, 1 (2024), 53–65. https: //www.vldb.org/pvldb/vol18/p53-yuan.pdf

    2024

  28. [41]

    Rajmohan, and Dongmei Zhang

    Yudong Liu, Xu Zhang, Shilin He, Hongyu Zhang, Liqun Li, Yu Kang, Yong Xu, Minghua Ma, Qingwei Lin, Yingnong Dang, S. Rajmohan, and Dongmei Zhang

  29. [42]

    Nur Zincir-Heywood, and Evangelos E

    Adetokunbo Makanju, A. Nur Zincir-Heywood, and Evangelos E. Milios. 2012. A Lightweight Algorithm for Message Type Extraction in System Application Logs. IEEE Transactions on Knowledge and Data Engineering 24, 11 (2012), 1921–1936. https://doi.org/10.1109/TKDE.2011.138

    2012 doi

  30. [44]

    Xinfu Liu, Yirui Wu, Yuting Zhou, Junyang Chen, Huan Wang, Ye Liu, and Shaohua Wan. 2024. Enhancing Large Language Models with Multimodality and Knowledge Graphs for Hallucination-free Open-set Object Recognition. Proceedings of the VLDB Endowment. ISSN 2150 (2024), 8097

    2024

  31. [45]

    Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, Huzefa Rangwala, Christos Faloutsos, and George Karypis. 2023. Which Ex- amples to Annotate for In-Context Learning? Towards Effective and Efficient Selection. arXiv:2310.20046 [cs.CL] https://arxiv.or...

    2023 arXiv

  32. [47]

    Masayoshi Mizutani. 2013. Incremental Mining of System Log Format. In 2013 IEEE International Conference on Services Computing . 595–602. https://doi.org/ 10.1109/SCC.2013.73

    2013 doi

  33. [48]

    Dasgupta, and Subhrajit Bhattacharya

    Animesh Nandi, Atri Mandal, Shubham Atreja, Gargi B. Dasgupta, and Subhrajit Bhattacharya. 2016. Anomaly Detection Using Program Control Flow Graph Mining From Execution Logs. In Proceedings of the 22nd ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mini...

    2016

  34. [49]

    Wim Martens, Matthias Niewerth, Tina Popp, Carlos Rojas, Stijn Vansum- meren, and Domagoj Vrgoč. 2023. Representing Paths in Graph Database Pattern Matching. Proc. VLDB Endow. 16, 7 (March 2023), 1790–1803. https: //doi.org/10.14778/3587136.3587151

    2023

  35. [50]

    Reham Omar, Ishika Dhall, Panos Kalnis, and Essam Mansour. 2023. A Universal Question-Answering Platform for Knowledge Graphs. Proceedings of the ACM on Management of Data 1 (2023), 1 – 25. https://api.semanticscholar.org/CorpusID: 257254920

    2023

  36. [51]

    OpenAI. 2024. Introducing gpt-4o: our fastest and most affordable flagship model. https://platform.openai.com/docs/guides/vision 2024-11-07

    2024

  37. [52]

    Abdelghny Orogat and Ahmed El-Roby. 2023. Maestro: Automatic Generation of Comprehensive Benchmarks for Question Answering Over Knowledge Graphs. Proc. ACM Manag. Data 1, 2, Article 177 (June 2023), 24 pages. https://doi.org/ 10.1145/3589322

    2023 doi

  38. [53]

    Zhencan Peng, Zhizhi Wang, and Dong Deng. 2023. Near-Duplicate Sequence Search at Scale for Large Language Model Memorization Evaluation. Proc. ACM Manag. Data 1, 2, Article 179 (June 2023), 18 pages. https://doi.org/10.1145/ 3589324

    2023

  39. [54]

    Sasho Nedelkoski, Jasmin Bogatinovski, Alexander Acker, Jorge Cardoso, and Odej Kao. 2021. Self-supervised Log Parsing. In Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track, Yuxiao Dong, Dunja Mladenić, and Craig Saunders (Eds.). Springer Intern...

    2021

  40. [55]

    Cedric Renggli, Xiaozhe Yao, Luka Kolar, Luka Rimanic, Ana Klimovic, and Ce Zhang. 2022. SHiFT: an efficient, flexible search engine for transfer learning. Proc. VLDB Endow. 16, 2 (Oct. 2022), 304–316. https://doi.org/10.14778/3565816. 3565831

    2022 doi

  41. [57]

    Jie Song and Yeye He. 2021. Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes. In Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD ’21). Association for Computing Machinery, New ...

    2021

  42. [58]

    Smith, and Tao Yu

    Hongjin SU, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu

  43. [60]

    Yushi Sun, Wang Jiachuan, Peng Cheng, Libin Zheng, Lei Chen, and Jian Yin

  44. [61]

    Yushi Sun, Hao Xin, and Lei Chen. 2023. RECA: Related Tables Enhanced Column Semantic Type Annotation Framework. Proc. VLDB Endow. 16, 6 (Feb. 2023), 1319–1331. https://doi.org/10.14778/3583140.3583149

    2023

  45. [62]

    Liang Tang, Tao Li, and Chang-Shing Perng. 2011. LogSig: generating system events from raw textual logs. In Proceedings of the 20th ACM International Con- ference on Information and Knowledge Management (Glasgow, Scotland, UK) (CIKM ’11). Association for Computing Machinery, N...

    2011

  46. [63]

    Yanni Tang, Zhuoxing Zhang, Kaiqi Zhao, Lanting Fang, Zhenhua Li, and Wu Chen. 2025. Substructure-Aware Log Anomaly Detection. Proc. VLDB Endow. 18, 2 (Feb. 2025), 213–225. https://doi.org/10.14778/3705829.3705840

    2025

  47. [64]

    Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https://qwenlm. github.io/blog/qwen2.5/

    2024

  48. [65]

    Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Çağatay Demiralp, Chen Chen, and Wang-Chiew Tan. 2022. Annotating Columns with Pre-trained Lan- guage Models. In Proceedings of the 2022 International Conference on Manage- ment of Data (Philadelphia, PA, USA) (SIGMOD ’22). ...

    2022 doi

  49. [66]

    The Apache Software Foundation. 2024. SparkR: R Front End for ’Apache Spark’. https://www.apache.orghttps://spark.apache.org R package version 3.5.1https://www.apache.org https://spark.apache.org

    2024

  50. [68]

    Tianzheng Wang, Ryan Johnson, and Ippokratis Pandis. 2017. Query fresh: log shipping on steroids. Proc. VLDB Endow. 11, 4 (Dec. 2017), 406–419. https: //doi.org/10.1145/3186728.3164137

    2017

  51. [69]

    Yuxiang Wang, Arijit Khan, Tianxing Wu, Jiahui Jin, and Haijiang Yan. 2020. Semantic Guided and Response Times Bounded Top-k Similarity Search over Knowledge Graphs . In 2020 IEEE 36th International Conference on Data En- gineering (ICDE) . IEEE Computer Society, Los Alamitos,...

    2020

  52. [70]

    Yubo Wang, Hao Xin, and Lei Chen. 2024. KGLink: A Column Type Annotation Method that Combines Knowledge Graph and Pre-Trained Language Model.2024 IEEE 40th International Conference on Data Engineering (ICDE) (2024), 1023–1035. https://api.semanticscholar.org/CorpusID:270214355

    2024

  53. [72]

    Fei Teng, Haoyang Li, and Lei Chen. 2025. LLMLog: Advanced Log Template Generation via LLM-driven Multi-Round Annotation. Online (2025). https: //github.com/XinTT/LLMLog

    2025

  54. [73]

    Zeyu Xiong, Daizong Liu, Xiang Fang, Xiaoye Qu, Jianfeng Dong, Jiahao Zhu, Keke Tang, and Pan Zhou. 2024. Rethinking Video Sentence Grounding From a Tracking Perspective With Memory Network and Masked Attention. IEEE Transactions on Multimedia 26 (2024), 11204–11218. https://d...

    2024

  55. [75]

    Lyu Xu, Byron Choi, Yun Peng, Jianliang Xu, and Sourav S Bhowmick. 2023. A Framework for Privacy Preserving Localized Graph Pattern Query Processing. Proc. ACM Manag. Data 1, 2, Article 129 (June 2023), 27 pages. https://doi.org/ 10.1145/3589274

    2023 doi

  56. [76]

    Chengcheng Yang, Lisi Chen, Hao Wang, Shuo Shang, Rui Mao, and Xiangliang Zhang. 2023. Dynamic Set Similarity Join: An Update Log Based Approach. IEEE Transactions on Knowledge and Data Engineering 35, 4 (2023), 3727–3741. https://doi.org/10.1109/TKDE.2021.3126631

    2023

  57. [77]

    Muzhi Yu, Zhaoxiang Lin, Jinan Sun, Runyun Zhou, Guoqiang Jiang, Hua Huang, and Shikun Zhang. 2022. TencentCLS: the cloud log service with high query performances. Proc. VLDB Endow. 15, 12 (Aug. 2022), 3472–3482. https://doi. org/10.14778/3554821.3554837

    2022

  58. [78]

    Chen Zhang, Sen Zhang, Chen Lei, and Peiguang Lin. 2018. Burstiness in Query Log: Web Search Analysis by Combining Global and Local Evidences. In 2018 IEEE 34th International Conference on Data Engineering (ICDE) . 1388–1391. https://doi.org/10.1109/ICDE.2018.00157

    2018

  59. [79]

    Anbiao Wu, Ye Yuan, Changsheng Li, Yuliang Ma, and Hao Zhang. 2024. At- tributed Network Embedding in Streaming Style. In 2024 IEEE 40th International Conference on Data Engineering (ICDE) . 3138–3150. https://doi.org/10.1109/ ICDE60146.2024.00243

    2024

  60. [80]

    Tianzhu Zhang, Han Qiu, Gabriele Castellano, Myriana Rifai, Chung Shue Chen, and Fabio Pianese. 2023. System Log Parsing: A Survey. IEEE Transactions on Knowledge and Data Engineering 35, 8 (2023), 8596–8614. https://doi.org/10. 1109/TKDE.2022.3222417

    2023

  61. [81]

    Junjielong Xu, Ruichun Yang, Yintong Huo, Chengyu Zhang, and Pinjia He

  62. [82]

    DivLog: Log Parsing with Prompt Enhanced In-Context Learning. 1–12. https://doi.org/10.1145/3597503.3639155

  63. [83]

    Xinyang Zhao, Xuanhe Zhou, and Guoliang Li. 2024. Chat2Data: An Interactive Data Analysis System with RAG, Vector Databases and LLMs. Proceedings of the VLDB Endowment 17, 12 (2024), 4481–4484

    2024

  64. [84]

    Weiguo Zheng, Lei Zou, Xiang Lian, Jeffrey Xu Yu, Shaoxu Song, and Dongyan Zhao. 2015. How to Build Templates for RDF Question/Answering: An Uncer- tain Graph Similarity Join Approach. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (Melbou...

    2015

  65. [85]

    Xiangmin Zhou, Chengkun He, Xi Chen, and Yanchun Zhang. 2024. HSAP: A Human-in-the-Loop Social Media-Based Situation Awareness Platform. Proc. VLDB Endow. 17, 12 (Aug. 2024), 4493–4496. https://doi.org/10.14778/3685800. 3685908

    2024 doi

  66. [86]

    Erkang Zhu, Silu Huang, and Surajit Chaudhuri. 2023. High-Performance Row Pattern Recognition Using Joins. Proc. VLDB Endow. 16, 5 (Jan. 2023), 1181–1195. https://doi.org/10.14778/3579075.3579090

    2023

  67. [87]

    Shaokun Zhang, Xiaobo Xia, Zhaoqing Wang, Ling-Hao Chen, Jiale Liu, Qingyun Wu, and Tongliang Liu. 2024. IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Models. In The Twelfth Inter- national Conference on Learning Representations . ...

    2024

  68. [88]

    Zhen Zhu, Yibo Wang, Shouqing Yang, Lin Long, Runze Wu, Xiu Tang, Junbo Zhao, and Haobo Wang. 2024. CORAL: Collaborative Automatic Labeling Sys- tem Based on Large Language Models. Proc. VLDB Endow. 17, 12 (Nov. 2024), 4401–4404. https://doi.org/10.14778/3685800.3685885 6 APPE...

    2024

  69. [89]

    Xinyi Zhang, Hong Wu, Yang Li, Zhengju Tang, Jian Tan, Feifei Li, and Bin Cui

  70. [90]

    An Efficient Transfer Learning Based Configuration Adviser for Database Tuning. Proc. VLDB Endow. 17, 3 (2023), 539–552. https://doi.org/10.14778/ 3632093.3632114

    2023

  71. [91]

    Xu Zhang, Yong Xu, Qingwei Lin, Bo Qiao, Hongyu Zhang, Yingnong Dang, Chunyu Xie, Xinsheng Yang, Qian Cheng, Ze Li, Junjie Chen, Xiaoting He, Randolph Yao, Jian-Guang Lou, Murali Chintalapati, Furao Shen, and Dongmei Zhang. 2019. Robust log-based anomaly detection on unstable ...

    2019

  72. [96]

    Jieming Zhu, Shilin He, Jinyang Liu, Pinjia He, Qi Xie, Zibin Zheng, and Michael R. Lyu. 2019. Tools and benchmarks for automated log parsing. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice (Montreal, Quebec, Canad...

    2019

  73. [2016]

    In Proceedings of the 38th International Conference on Software Engineering Compan- ion (Austin, Texas) (ICSE ’16)

    Log clustering based problem identification for online service systems. In Proceedings of the 38th International Conference on Software Engineering Compan- ion (Austin, Texas) (ICSE ’16). Association for Computing Machinery, New York, NY, USA, 102–111. https://doi.org/10.1145/...

  74. [2021]

    In 2021 IEEE 37th International Conference on Data Engineering (ICDE)

    LogLog Filter: Filtering Cold Items within a Large Range over High Speed Data Streams. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). 804–815. https://doi.org/10.1109/ICDE51399.2021.00075

    2021

  75. [2022]

    https://api.semanticscholar.org/CorpusID: 246822534

    UniParser: A Unified Log Parser for Heterogeneous Log Data.Proceedings of the ACM Web Conference 2022 (2022). https://api.semanticscholar.org/CorpusID: 246822534

    2022

  76. [2023]

    In The Eleventh International Conference on Learning Representations

    Selective Annotation Makes Language Models Better Few-Shot Learners. In The Eleventh International Conference on Learning Representations . https: //openreview.net/forum?id=qY1hlv7gwg

  77. [2024]

    Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models. Proc. VLDB Endow. 18, 1 (2024), 42–52. https://www.vldb.org/pvldb/vol18/p42-jiang.pdf

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.