REVIEW 3 major objections 3 minor 37 references
From Few-Label to Zero-Label: An Approach for Cross-System Log-Based Anomaly Detection with Meta-Learning
T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read FreeLog detects log anomalies across systems with zero target labels
desk verdict FreeLog combines meta-learning and domain adaptation for zero-label log anomaly detection, but the evaluation omits the key unsupervised LogTAD baseline and the headline F1 claim is inaccurate. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is FreeLog's system-agnostic representation meta-learning network, made of a feature extractor $f_{\theta_e}$, an anomaly classifier $f_{\theta_\omega}$, and a domain classifier $f_{\theta_d}$. The mechanism is a min-max objective: the domain classifier maximizes its ability to separate source from target while the feature extractor minimizes that ability, forcing the learned features to be domain-invariant, while the anomaly classifier minimizes source classification loss. Meta-learning then treats each source-target split as a meta-task, adapting the extractor on support sets and meta-optimizing it on query sets, so the aligned representation generalizes to the target at inference. Logs are parsed by Drain and embedded in a shared global space via pretrained word embeddings, so events from different systems are comparable.
What would settle it
Take a source-target pair with minimal log vocabulary overlap (for example, train on HDFS logs and test on an application-server log set), run FreeLog with no target labels, and compare its F1 against a trivial majority-class predictor. If the score falls to chance, the shared-semantics premise fails; alternatively, ablate the domain classifier and check whether F1 drops to the level of MetaLog without target labels.
Extended reading notes
Core claim
The central claim is that a system-agnostic representation, learned from labeled source logs and unlabeled target logs, is enough to detect anomalies in the target system without a single target label. FreeLog combines a meta-learning objective with an adversarial domain classifier: the feature extractor is trained so that the domain classifier cannot tell source from target features, while the anomaly classifier learns normal-versus-anomalous boundaries from source labels. Meta-tasks partition source and target data into support and query sets, and meta-optimization updates the extractor to adapt quickly to the target. Reported F1 scores across the four source-target combinations are 86.01, 80.61, 80.21, and 77.55, which the paper presents as comparable to the few-label state of the art.
Load-bearing premise
The load-bearing premise is that source and target logs, after Drain parsing and shared semantic embeddings, are similar enough that aligning their feature distributions with unlabeled target data also aligns the normal-versus-anomaly boundary; if a target system's anomaly patterns are absent from the source, zero-label transfer has no signal to find them.
Editorial extensions
If this is right
- Zero-label transfer becomes practical: new systems can be monitored without labeling any target logs, provided unlabeled target logs are available during training.
- FreeLog matches or beats few-label baselines on the four reported transfers, suggesting the annotation budget for cross-system log anomaly detection can be cut to zero in similar settings.
- The method outperforms direct zero-shot and transfer-learning baselines, indicating that the combination of adversarial alignment and meta-learning, not just source training, drives the gain.
- Because the approach is transductive, it applies when target logs exist in bulk but labels are missing, not to a target system that appears only at inference time.
- The results depend on shared semantics after Drain parsing and pretrained embeddings, so pairings with similar log vocabulary should transfer best.
Reading between the lines
- A direct test of the transferability premise: hold out a target system whose log event vocabulary barely overlaps the source (for example, source HDFS with a web-server log set) and check whether F1 stays above 80 percent; if it drops to chance, the shared-semantics assumption is the bottleneck.
- The adversarial alignment uses all unlabeled target logs equally; if the target's anomaly rate or fault types differ sharply from the source, the transferred boundary may need a target-specific threshold or calibration, which the paper does not explore.
- Since the meta-learning update is model-agnostic meta-learning, FreeLog could likely be extended to streaming settings where unlabeled target logs arrive over time and the feature extractor keeps adapting, though the paper only tests the static case.
- The three datasets share cluster-system vocabulary; testing on a different domain, such as mobile or application-server logs, would reveal whether 'system-agnostic' holds beyond datacenter-style logs.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FreeLog, a meta-learning and adversarial domain adaptation method for cross-system log-based anomaly detection in a setting where the target system has no labeled logs. The method uses a feature extractor trained with a source-domain classification loss and a domain-adversarial loss between labeled source and unlabeled target logs, and is evaluated on four source-target transfers among HDFS, BGL, and OpenStack. The authors report F1 scores around 80% and claim this is the first zero-label cross-system method and that its performance is comparable to few-label state-of-the-art methods.
Significance. If the central claims hold, the paper addresses a practically important cold-start problem by removing the need for any target labels. Strengths include the use of three public benchmark datasets, comparison against several baseline families (semi-supervised, unsupervised, zero-shot, transfer learning, and meta-learning), and a clearly stated problem setting. The main significance is conditional: the zero-label claim depends on a fair comparison with the original unsupervised LogTAD baseline, which is missing, and the headline F1 claim is not literally supported by the reported table. The method description also lacks the details needed to reproduce or independently verify the reported numbers.
major comments (3)
- [Section 3, Block (d) and Table 1] The most relevant prior baseline for the paper's central claim is LogTAD [13], which is titled 'Unsupervised Cross-system Log Anomaly Detection via Domain Adaptation' and therefore operates in the same zero-label setting as FreeLog (labeled source, unlabeled target). However, in Block (d) of Table 1, LogTAD is supplied with 30% normal and 1% anomalous target labels, converting it into a few-label method. The original unsupervised LogTAD is never evaluated. Without this baseline, the evidence cannot distinguish whether FreeLog's gains come from the proposed meta-learning/adversarial alignment or from access to large amounts of unlabeled target data, and the claims of being 'the first' and 'comparable to SOTA' are not supported against the directly relevant prior work. The authors should run original LogTAD in the zero-label setting and report results.
- [Abstract and Section 1] The abstract and Section 1 state that 'under zero-label conditions, FreeLog achieves an F1-score exceeding 80%'. Table 1 shows FreeLog's F1 on OpenStack to BGL is 77.55, so the claim is false as literally written. If the intended meaning is that the average F1 exceeds 80%, that should be stated explicitly, and the abstract should be corrected to match the reported numbers.
- [Section 2.2 and Section 3] The paper does not provide exact definitions of the losses L_c and L_ad, the architecture of the feature extractor, anomaly classifier, and domain classifier, the inner-loop and meta-step sizes, the values of beta and gamma, the number of meta-tasks, or the target-data split proportions mentioned in Section 3. Table 1 also reports single precision/recall/F1 values without variance or significance tests. Because the central empirical claim is that FreeLog is comparable to few-label SOTA, the absence of these details and of any statistical assessment makes the claim difficult to verify. Please report the missing experimental settings and at least standard deviations over multiple runs.
minor comments (3)
- [Figure 1] Figure 1 contains garbled symbols (e.g., 'ௌ', 'ࣸࣵ') that appear to be font or encoding artifacts; the figure should be regenerated so all mathematical and textual elements are legible.
- [Section 3, Table 1] The rows 'BGL to HDFS' and 'OpenStack to HDFS' show identical Block (a) and Block (b) numbers, and likewise 'HDFS to BGL' and 'OpenStack to BGL' are identical in those blocks. This is expected because Block (a) trains on target data only, but the duplication should be explained or the rows merged to avoid confusion.
- [Section 2.1] The term 'zero-label' should be defined precisely at first use: the method assumes unlabeled target logs are available during training, so 'zero-label' means no target labels, not no target data. This transductive setting should be stated in the abstract or introduction to prevent overgeneralization.
Circularity Check
No significant circularity; the zero-label performance is empirical and externally benchmarked, though the novelty claim overstates novelty by omitting the original unsupervised LogTAD baseline.
full rationale
FreeLog's central empirical claims are not derived from its own fitted parameters: the anomaly classifier is trained on labeled source logs, the domain classifier aligns source and target features using unlabeled target logs, and the reported F1 scores are measured on held-out public log datasets (HDFS, BGL, OpenStack) against several baselines. No equation in Section 2.2 reduces the target-label prediction to the training objective by construction, because target labels are never used in any loss term. The only self-citation is the adoption of MetaLog's global-consistency semantic embedding ('we adopt the global consistency semantic embedding approach inspired by MetaLog [32]'); this is an implementation choice, not a load-bearing uniqueness argument, and it does not by itself force the zero-label result. The most serious concern is the evaluation of LogTAD: Ref. [13] is titled 'Unsupervised Cross-system Log Anomaly Detection via Domain Adaptation', yet in Block (d) LogTAD is given target labels (30% normal and 1% anomalous sessions), so the original unsupervised LogTAD is never tested. That omission undermines the 'first' and 'comparable to SOTA' novelty claims, but it is a prior-art/comparison-fairness problem rather than circularity: the FreeLog F1 numbers are not constructed from the baseline numbers. Hence the circularity score is 1.
Assumptions & free parameters
free parameters (6)
- beta (domain adversarial loss weight) =
Not reported.
- gamma (classification loss weight) =
Not reported.
- alpha (meta step size) =
Not reported.
- delta (inner-loop learning rate) =
Not reported.
- Drain parsing parameters =
Not reported.
- Target data split proportions =
Not reported.
assumptions (4)
- domain assumption A domain-invariant feature space exists between source and target log systems, so adversarial alignment from unlabeled target data suffices for anomaly classification.
- domain assumption Drain parsing plus pretrained word embeddings produce semantically consistent log event representations across systems.
- domain assumption Meta-learning over synthetic meta-tasks from source and target unlabeled data generalizes to the target's anomaly distribution.
- domain assumption MAML-style gradient-based meta-optimization converges in this low-resource, adversarial setting.
Cite this review
Pith. "Pith review of From Few-Label to Zero-Label: An Approach for Cross-System Log-Based Anomaly Detection with Meta-Learning." pith.science (2026). https://pith.science/paper/4NRNCXZW
@misc{pith2026250719806,
author = {Pith},
title = {Pith review of: From Few-Label to Zero-Label: An Approach for Cross-System Log-Based Anomaly Detection with Meta-Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/4NRNCXZW}},
note = {Machine review of arXiv:2507.19806}
}
read the original abstract
Log anomaly detection plays a critical role in ensuring the stability and reliability of software systems. However, existing approaches rely on large amounts of labeled log data, which poses significant challenges in real-world applications. To address this issue, cross-system transfer has been identified as a key research direction. State-of-the-art cross-system approaches achieve promising performance with only a few labels from the target system. However, their reliance on labeled target logs makes them susceptible to the cold-start problem when labeled logs are insufficient. To overcome this limitation, we explore a novel yet underexplored setting: zero-label cross-system log anomaly detection, where the target system logs are entirely unlabeled. To this end, we propose FreeLog, a system-agnostic representation meta-learning method that eliminates the need for labeled target system logs, enabling cross-system log anomaly detection under zero-label conditions. Experimental results on three public log datasets demonstrate that FreeLog achieves performance comparable to state-of-the-art methods that rely on a small amount of labeled data from the target system.
Figures
Reference graph
Works this paper leans on
-
[13]
Xiao Han and Shuhan Yuan. 2021. Unsupervised Cross-system Log Anomaly Detection via Domain Adaptation. InProceedings of the 30th ACM International Conference on Information & Knowledge Management(Virtual Event, Queensland, Australia)(CIKM ’21). Association for Computing Machinery, New York, NY, USA, 3068–3072. doi:10.1145/3459637.3482209
arXiv 2021
-
[1]
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Mach. Learn.79, 1–2 (May 2010), 151–175. doi:10.1007/s10994-009-5152-4
-
[2]
Rui Chen, Shenglin Zhang, Dongwen Li, Yuzhe Zhang, Fangrui Guo, Weibin Meng, Dan Pei, Yuzhi Zhang, Xu Chen, and Yuqing Liu. 2020. LogTransfer: Cross- System Log Anomaly Detection for Software Systems with Transfer Learning. In2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE). 37–47. doi:10.1109/ISSRE5003.2020.00013
arXiv 2020
-
[3]
Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. 2017. DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA)(CCS ’17). Association for Computing Machinery, New York, NY, USA, 1285–1298. doi:10.1145/3133956.3134015
arXiv 2017
-
[4]
Chiming Duan, Tong Jia, Huaqian Cai, Ying Li, and Gang Huang. 2023. Afalog: A general augmentation framework for log-based anomaly detection with ac- tive learning. In2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 46–56
work page 2023
-
[5]
Chiming Duan, Tong Jia, Ying Li, and Gang Huang. 2023. Aclog: An approach to detecting anomalies from system logs with active learning. In2023 IEEE International Conference on Web Services (ICWS). IEEE, 436–443
work page 2023
- [6]
-
[7]
Chiming Duan, Tong Jia, Yong Yang, Guiyang Liu, Jinbu Liu, Huxing Zhang, Qi Zhou, Ying Li, and Gang Huang. 2025. EagerLog: Active Learning Enhanced Retrieval Augmented Generation for Log-based Anomaly Detection. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5
work page 2025
Show all 37 references
-
[8]
Chiming Duan, Yong Yang, Tong Jia, Guiyang Liu, Jinbu Liu, Huxing Zhang, Qi Zhou, Ying Li, and Gang Huang. 2025. FAMOS: Fault diagnosis for Microservice Systems through Effective Multi-modal Data Fusion. In2025 IEEE/ACM 47th International Conference on Software Engineering (IC...
2025
-
[9]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta- Learning for Fast Adaptation of Deep Networks. InProceedings of the 34th In- ternational Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh...
2017
-
[10]
Bastiaan Kleijn, and Mengjie Zhang
Muhammad Ghifary, David Balduzzi, W. Bastiaan Kleijn, and Mengjie Zhang
-
[11]
Micah Goldblum, Liam Fowl, and Tom Goldstein. 2020. Adversarially Robust Few- Shot Learning: A Meta-Learning Approach. InAdvances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., ...
2020
-
[12]
Jiatao Gu, Yong Wang, Yun Chen, Victor O. K. Li, and Kyunghyun Cho. 2018. Meta-Learning for Low-Resource Neural Machine Translation. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun...
2018 doi
-
[15]
Minghua He, Tong Jia, Chiming Duan, Huaqian Cai, Ying Li, and Gang Huang
-
[16]
Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. 2017. Drain: An Online Log Parsing Approach with Fixed Depth Tree. In2017 IEEE International Conference on Web Services (ICWS). 33–40. doi:10.1109/ICWS.2017.13
2017 doi
-
[17]
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2022. Meta-Learning in Neural Networks: A Survey.IEEE Transactions on Pattern Analysis and Machine Intelligence44, 9 (2022), 5149–5169. doi:10.1109/TPAMI. 2021.3079209
2022
-
[18]
Tong Jia, Ying Li, Yong Yang, Gang Huang, and Zhonghai Wu. 2022. Augmenting Log-based Anomaly Detection Models to Reduce False Anomalies with Human Feedback. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(Washington DC, USA)(KDD ’22). As...
2022 doi
-
[19]
Jinhan Kim, Valeriy Savchenko, Kihyuck Shin, Konstantin Sorokin, Hyunseok Jeon, Georgiy Pankratenko, Sergey Markov, and Chul-Joo Kim. 2020. Auto- matic abnormal log detection by analyzing log history for providing debugging insight. InProceedings of the ACM/IEEE 42nd Internati...
2020
-
[20]
Van-Hoang Le and Hongyu Zhang. 2021. Log-based Anomaly Detection With- out Log Parsing. In2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 492–504. doi:10.1109/ASE51524.2021.9678773
2021
-
[21]
Weibin Meng, Ying Liu, Yichen Zhu, Shenglin Zhang, Dan Pei, Yuqing Liu, Yihao Chen, Ruizhi Zhang, Shimin Tao, Pei Sun, and Rong Zhou. 2019. LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstruc- tured Logs. InProceedings of the Twenty-Eighth In...
2019 doi
-
[22]
Adam Oliner and Jon Stearley. 2007. What Supercomputers Say: A Study of Five System Logs. In37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’07). 575–584. doi:10.1109/DSN.2007.103
2007 doi
-
[23]
Munawar, and Paul A.S
Thomas Reidemeister, Mohammad A. Munawar, and Paul A.S. Ward. 2010. Identi- fying symptoms of recurrent faults in log files of distributed information systems. In2010 IEEE Network Operations and Management Symposium - NOMS 2010. 187–194. doi:10.1109/NOMS.2010.5488459
2010
-
[24]
Richa Upadhyay, Ronald Phlypo, Rajkumar Saini, and Marcus Liwicki. 2024. Sharing to Learn and Learning to Share; Fitting Together Meta, Multi-Task, and Transfer Learning: A Meta Review.IEEE Access12 (2024), 148553–148576. doi:10.1109/ACCESS.2024.3478805
2024
-
[25]
Jun Wu and Jingrui He. 2022. A Unified Meta-Learning Framework for Dy- namic Transfer Learning. InProceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, Lud De Raedt (Ed.). Interna- tional Joint Conferences on Artificial Intelligen...
2022 doi
-
[26]
Wensheng Xia, Ying Li, Tong Jia, and Zhonghai Wu. 2019. BugIdentifier: An Approach to Identifying Bugs via Log Mining for Accelerating Bug Reporting Stage. In2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS). 167–175. doi:10.1109/QRS.2...
2019
-
[27]
Pei Xiao, Tong Jia, Chiming Duan, Huaqian Cai, Ying Li, and Gang Huang. 2024. LogCAE: An Approach for Log-based Anomaly Detection with Active Learning and Contrastive Learning. In2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 144–155
2024
-
[28]
Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael I. Jordan. 2009. Detecting large-scale system problems by mining console logs. InProceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles(Big Sky, Montana, USA)(SOSP ’09). Association for Computi...
2009
-
[29]
Lin Yang, Junjie Chen, Zan Wang, Weijing Wang, Jiajun Jiang, Xuyuan Dong, and Wenbin Zhang. 2021. Semi-Supervised Log-Based Anomaly Detection via Probabilistic Label Estimation. In2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 1448–1460. doi:10.110...
2021
-
[30]
Xixuan Yang, Xin Huang, Chiming Duan, Tong Jia, Shandong Dong, Ying Li, and Gang Huang. 2025. Enhancing Web Service Anomaly Detection via Fine- grained Multi-modal Association and Frequency Domain Analysis.arXiv preprint arXiv:2501.16875(2025)
2025 arXiv
-
[31]
Kun Yin, Meng Yan, Ling Xu, Zhou Xu, Zhao Li, Dan Yang, and Xiaohong Zhang
-
[32]
Chenyangguang Zhang, Tong Jia, Guopeng Shen, Pinyan Zhu, and Ying Li. 2024. MetaLog: Generalizable Cross-System Anomaly Detection from Logs with Meta- Learning. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering(Lisbon, Portugal)(ICSE ’24). Ass...
2024
-
[33]
Xu Zhang, Yong Xu, Qingwei Lin, Bo Qiao, Hongyu Zhang, Yingnong Dang, Chunyu Xie, Xinsheng Yang, Qian Cheng, Ze Li, Junjie Chen, Xiaoting He, Ran- dolph Yao, Jian-Guang Lou, Murali Chintalapati, Furao Shen, and Dongmei Zhang
-
[2017]
doi:10.1109/TPAMI.2016.2599532
Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization.IEEE Transactions on Pattern Analysis and Machine Intelligence39, 7 (2017), 1414–1430. doi:10.1109/TPAMI.2016.2599532
2017
-
[2019]
Robust log-based anomaly detection on unstable log data. InProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(Tallinn, Estonia) (ESEC/FSE 2019). Association for Computing Machinery...
2019
-
[2020]
In2020 IEEE International Conference on Software Maintenance and Evolution (ICSME)
Improving Log-Based Anomaly Detection with Component-Aware Analysis. In2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). 667–671. doi:10.1109/ICSME46990.2020.00069
2020
-
[2024]
In2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE)
LLMeLog: An Approach for Anomaly Detection based on LLM-enriched Log Events. In2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 132–143
-
[2025]
In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE)
Weakly-supervised Log-based Anomaly Detection with Inexact Labels via Multi-instance Learning. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, 726–726
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.