Pith. sign in

REVIEW 4 major objections 6 minor 53 references

CAPTAIN detects APT attacks by scoring log entries with a context-conditioned language model, achieving an average AUC of 0.929 on minimally preprocessed ATLAS logs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 09:13 UTC pith:YRBDVRRQ

load-bearing objection A plausible new detector with a genuinely useful AIRTAG replication study, but the headline robustness is partly an artifact of inverted AUC and a centered smoother; worth refereeing after the evaluation is tightened. the 4 major comments →

arxiv 2607.20832 v1 pith:YRBDVRRQ submitted 2026-07-23 cs.LG cs.AIcs.CLcs.CR

Beyond Heavy Log Curation: Perplexity-Based APT Detection via Unsupervised, Context-Augmented Language Models

Shoya Otsu , Kei Suzuki , Toshiaki Koike-Akino , Jing Liu , Ye Wang This is my paper
classification cs.LG cs.AIcs.CLcs.CR
keywords advanced persistent threatslog anomaly detectionperplexityunsupervised learninglanguage modelscontext augmentationQ-FormerATLAS dataset
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

CAPTAIN is a new detector for advanced persistent threats (APTs) that asks a pre-trained language model to predict each log entry and uses the resulting perplexity—how surprised the model is—as an anomaly score. The paper's central claim is that this perplexity score, when conditioned on recent log history through a Q-Former bridge, reliably separates malicious from benign entries even when logs are long, noisy, and preprocessed with generic, domain-agnostic steps. On the ATLAS benchmark with minimal preprocessing, CAPTAIN achieves an average AUC of 0.929 versus 0.684 for the strong AIRTAG baseline, and it stays stable when input length grows, whereas AIRTAG degrades. The authors also re-evaluate AIRTAG and find that two evaluation artifacts—label markers left in training data and a post-filter that used ground-truth labels—inflated its reported performance. If correct, CAPTAIN would let defenders deploy attack detection without building and maintaining costly, dataset-specific preprocessing pipelines.

Core claim

The paper's core discovery is that a decoder-only language model, augmented with a compact context summary of recent log entries via an encoder and a Q-Former bridge, can score log entries by perplexity in a way that is robust to input length and preprocessing quality. The authors show that the context bridge provides a structural benefit beyond simple domain adaptation of the language model: removing the Q-Former and fine-tuning a vanilla LM on benign logs does not match CAPTAIN's performance, and replacing the Q-Former with MLP or SwiGLU adapters degrades AUC. They further show that smoothing the perplexity time series with a Wiener filter consistently improves detection (from 0.889 to 0.9

What carries the argument

The key machinery is a two-stage Q-Former bridge that compresses the current log entry and the recent history (previous C entries) into a fixed set of learned query tokens, which are injected into a decoder-only LM as soft context tokens. The LM computes the conditional perplexity of the current entry given that context. A Wiener filter then smooths the perplexity time series, treating it as a Markov-correlated signal, to stabilize the anomaly score. The bridge's cross-attention selects history fragments relevant to the current entry, which the authors show outperforms simple MLP or SwiGLU projections.

Load-bearing premise

The load-bearing premise is that the direction of anomaly is known without labels: when AUC falls below 0.5 (as on S2), the evaluation simply reports 1−AUC, which assumes an oracle tells the detector to invert its decision rule. If that oracle is unavailable in practice, the detector would systematically flag the wrong class.

What would settle it

On the S2 split, CAPTAIN's raw AUC is 0.254 (reported as 1−AUC 0.746), meaning benign entries generally receive higher perplexity than malicious ones. If a deployment runs CAPTAIN on similar unseen-benign-shift logs without knowing to invert, the detector will mark the wrong entries; observing this on any real dataset without an inversion oracle would falsify the claim that high perplexity reliably identifies attacks.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, APT detection no longer requires heavy, dataset-specific preprocessing; generic text normalization suffices, cutting development and operational cost.
  • Context-conditioned perplexity should improve detection of multi-step attacks whose individual entries look benign in isolation but are anomalous in sequence.
  • Smoothing the perplexity series converts a noisy per-entry signal into a stable detector, which should generalize to other log-anomaly settings.
  • Because lightweight decoder LMs (around 0.6B parameters) are sufficient, detection can run per-host in near-real time rather than in a central aggregation pipeline.
  • The paper's own S2 result shows that benign distribution shifts can produce high perplexity, so deployments would need threshold recalibration or human-in-the-loop triage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 1−AUC inversion on S2 relies on an oracle knowing that the scoring direction is flipped; without such an oracle in deployment, the reported average AUC may overstate the detector's robustness. A practical system would need a separate calibration mechanism to decide when 'low perplexity' is the anomaly.
  • The same Q-Former context-bridging design could transfer to other anomaly-detection domains where the current observation only makes sense relative to recent history, such as network-flow volume or system-call sequences.
  • An adversary aware of the perplexity scoring could attempt to craft log entries that look idiomatic under the benign distribution; the paper does not test adversarial evasion, so the method's security under an adaptive attacker remains an open question.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper revisits the AIRTAG APT-detection baseline, identifies label artifacts and cross-split keyword overlap in its evaluation, and proposes CAPTAIN, an unsupervised detector that scores each log entry by the perplexity of a decoder-only LM conditioned on compact context tokens produced by a DistilBERT encoder and a two-stage Q-Former bridge. A Wiener filter is applied to the resulting perplexity time series. The main experiments compare CAPTAIN with AIRTAG on ATLAS under both AIRTAG-style preprocessing and a minimally processed version; the headline result is an average AUC of 0.929 on the minimally processed dataset versus AIRTAG's 0.684. Ablations cover the Q-Former bridge, context window size, alternative adapters, decoder backbone, and Wiener filtering.

Significance. The paper addresses a relevant operational problem, and it contains several valuable pieces: a careful replication of AIRTAG documenting that its post-filter relies on label markers, a keyword-sanitization protocol intended to reduce cross-split memorization, and a coherent unsupervised training scheme for a context-prompted perplexity scorer. If the headline robustness result survives a corrected evaluation, the contribution is significant: it would show that lightweight pretrained LMs can operate on long, minimally curated logs without domain-specific feature engineering, which is a practical improvement over graph-heavy or heavily curated pipelines. The external benchmark evaluation means the core comparison is not circular by construction. However, the evaluation protocol currently overstates the result: the S2 AUC inversion and the centered Wiener filter both inject information unavailable at inference time, and no uncertainty quantification is provided.

major comments (4)
  1. [§4.1 / Table 4] Reporting 1−AUC for S2 is not a valid deployment metric. On S2, the raw CAPTAIN AUC is 0.254 and the raw AIRTAG AUC is 0.311; after inversion these become 0.746 and 0.689. Consequently, the Experiment 2 average drops from 0.929 to 0.880 if the un-inverted S2 value is used, and CAPTAIN no longer outperforms AIRTAG on all splits. The inversion also contradicts the paper's core premise (Section 3.1, Eq. 4) that attack entries have higher perplexity; Section 5.3 explicitly concedes that on S2 benign entries receive higher perplexity than malicious ones. Since no label-free mechanism for choosing the inversion direction is provided, the headline average is inflated. Please report un-inverted AUCs, and treat direction reversal as a separate method with its own calibration procedure rather than as a free correction.
  2. [§3.4, Eq. (5)] The Wiener filter is centered: with c=(L−1)/2, the smoothed score at time t uses PPL values up to t+(L−1)/2, i.e., future observations. The stated use case is 'fast detection on each host' (Section 3.3), but this filter requires waiting for future log entries. Because attack labels are strongly autocorrelated (Appendix B, self-transition 0.972–0.996), future PPL values are highly informative, and the reported 0.889→0.929 improvement from smoothing (Section 4.4.5) may be largely a lookahead artifact. Moreover, the filter parameters L, p, and σ² are never disclosed. Please evaluate with a causal filter and report the parameter values; if the centered filter is retained, quantify the detection latency and show results for a causal variant.
  3. [§4.4.2 / Table 4] The main experimental configuration is under-specified. The context-size ablation finds size 1 best (AUC 0.886), but the manuscript does not state which C is used in the main results of Table 4, nor how many Q-Former query tokens are used. The Wiener filter parameters L, p, and σ² are likewise absent. Without these values, the main results and the smoothing ablation cannot be reproduced or audited. Please specify all hyperparameters for the reported results and, ideally, release the configuration and code.
  4. [§4 / Figures 5–6] No error bars, repeated runs, or significance tests are reported. Single-run AUC differences are used to draw conclusions about context size and decoder backbone (e.g., 0.900 vs 0.899 vs 0.887 in Figure 5d; 0.886 vs 0.872 vs 0.871 in Figure 5b). These differences may be within run-to-run noise. Please report multiple seeds, confidence intervals, or appropriate statistical tests, especially for the smoothing gain and the Q-Former-versus-MLP comparison.
minor comments (6)
  1. [Abstract] '...remains robust under substantially less curated inputs, that reduces the development and operational cost...' — the clause is grammatically awkward; consider 'which reduces' or rephrasing.
  2. [§2.3] The sentence 'by observing that removing these "+/-" labels significantly degraded performance. demonstrating that it no longer performed its intended function.' begins with a lowercase letter and is a fragment; it should be joined to the preceding sentence.
  3. [Figure 5 / Table 4] The asterisk convention for 'S2*' is defined only in the Table 4 note; the figure captions should restate that starred values are inverted AUCs.
  4. [§4.4.1] The heading text 'we evaluate the contribution...' begins with a lowercase 'we'; capitalize for consistency.
  5. [Table 2] The caption 'Malicious Keyword Mapping (left→right)' is confusing; clearer column labels such as 'Training-split replacement → Test-split original' would help.
  6. [Appendix C] The claim that 'more than half of the entries are dominated by logs that appear almost exclusively in S2' should be quantified precisely; provide exact overlap counts with the strings listed in Table 6.

Circularity Check

1 steps flagged

S2 AUC reported as 1−AUC makes the headline 0.929 partly an oracle-inverted score; otherwise evaluation is external.

specific steps
  1. fitted input called prediction [Section 4.1 (Performance metric; Table 4 note) and Section 5.3]
    "Since an AUC below 0.5 indicates that the scoring function is negatively correlated with the ground-truth labels (i.e., reversing the decision rule would yield better-than-random discrimination), we report the corresponding discriminative ability using 1−AUC for those cases."

    CAPTAIN's stated decision rule (Section 3.1, Eq. 4) is: 'x is suspicious ⇔ PPL(x|c)>τ'. Table 4 reports CAPTAIN's Experiment 2 AUC on S2 as *0.746. Since the note says values below 0.5 are inverted, the actual AUC was 0.254, so the reported 0.746 is the AUC of the reversed rule PPL<τ, chosen after seeing the labels. This inverted S2 value is then included in the average 0.929 used to claim robustness on minimally preprocessed logs. Section 5.3 admits 'on S2 we observe an AUC below 0.5, which indicates that benign entries can receive higher perplexity than malicious ones in that split.' Thus the S2 'prediction' is not the performance of the proposed CAPTAIN detector but of its label-chosen inverse, making that part of the headline robustness result reduce to the evaluation choice by constru

full rationale

The central derivation of CAPTAIN is not circular: it trains a Q-Former bridge and decoder LM with an unsupervised reconstruction objective on benign logs, and the main comparison against AIRTAG on the ATLAS benchmark is external. Ablations (vanilla LM, context size, adapter replacement, LM backbone) are measured with held-out data and do not reduce to the inputs. The one construction-dependent component is S2: reporting 1−AUC turns an anti-correlated score into a success and is included in the average AUC, so the headline 0.929 is partly an oracle-inverted number. The Wiener filter is also a concern: Eq. (5) uses a centered window with c=(L−1)/2, so future PPL values contribute to the current smoothed score, and the filter parameters p, σ², L are never reported; with strongly autocorrelated attack labels (Appendix B), this may inflate the smoothing gain. However, that is a lookahead/evaluation risk rather than a definitional circularity, so it is weighed in the rationale but not scored as a circular step.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The method rests on the domain assumption that malicious logs are surprising to a benign-trained LM, plus several unreported hyperparameters (smoothing filter, query count, context size). No new physical entities are introduced.

free parameters (5)
  • Wiener filter window length L = not reported
    Controls temporal span of PPL smoothing; value is never stated in Section 3.4 or 4.4.5.
  • Wiener filter crossover probability p = not reported
    Sets correlation decay in Eq. 7-8; never specified.
  • Wiener filter noise variance σ² = not reported
    Diagonal loading in Eq. 7; never specified.
  • Q-Former number of learned query tokens = not reported
    Fixed number of context tokens injected into the LM; the count is not given.
  • Context window size C (number of past log entries) = not reported for main results
    Ablation tests 0, 1, 5, 9 with best at 1, but the main CAPTAIN results do not state which C is used.
axioms (4)
  • domain assumption Benign-only LM perplexity is higher for attack-related log entries than for normal entries.
    Core detection assumption in Section 3.1; contradicted on S2, where AUC fell below 0.5.
  • domain assumption ATLAS's synthetic attack labels and log distribution are representative of real APT logs.
    Evaluation and training rely entirely on ATLAS; acknowledged in Limitations.
  • ad hoc to paper Autoencoder-style reconstruction of LM input is a sufficient training signal for the Q-Former context bridge.
    No formal justification; only ablations suggest it helps.
  • ad hoc to paper PPL time series follows the assumed Markov correlation structure in Eq. 7-8.
    Wiener filter design assumes this; parameters are not estimated from data or reported.

pith-pipeline@v1.3.0-alltime-deepseek · 21234 in / 10874 out tokens · 126384 ms · 2026-08-01T09:13:48.226441+00:00 · methodology

0 comments
read the original abstract

Advanced Persistent Threats (APTs) remain difficult to detect because only a small fraction of events in large-scale logs are attack-related, and investigation is expensive and hard to scale. Prior machine-learning approaches can reduce analyst workload, but they often rely on heavily curated training data and sophisticated preprocessing pipelines. Building and maintaining such pipelines require substantial domain expertise and engineering cost. Motivated by insights from a study of a strong APT detection baseline, we propose CAPTAIN (Context-Augmented Perplexity-based Threat Activity log detectIoN), a perplexity-based detector that leverages general, pre-trained language models with minimal, domain-agnostic preprocessing, enabling robust scoring of long, minimally processed log entries. CAPTAIN encodes recent history with an encoder model and a Q-Former-style bridge, then injects the compact context tokens into the decoder input so that perplexity reflects temporal context. To improve stability, CAPTAIN additionally applies smoothing filters to the perplexity time series. Across APT-oriented benchmarks, CAPTAIN competes with strong existing baselines and remains robust under substantially less curated inputs, that reduces the development and operational cost of advanced log preprocessing.

Figures

Figures reproduced from arXiv: 2607.20832 by Jing Liu, Kei Suzuki, Shoya Otsu, Toshiaki Koike-Akino, Ye Wang.

Figure 1
Figure 1. Figure 1: CAPTAIN overview. CAPTAIN ingests minimally processed, time-ordered logs (the current entry and its recent history). A text encoder summarizes the history, and a Q-Former bridges it into a decoder-only LM as context tokens. The LM assigns a perplexity (PPL) score to each entry. Then, the PPL time series is smoothed in post-processing, and entries with high final scores are flagged as suspicious for analyst… view at source ↗
Figure 2
Figure 2. Figure 2: CAPTAIN architecture with Q-Former details. The text encoder embeds the current log entry and log history. Q-Former uses a fixed set of learned query tokens and updates them with Transformer blocks (self-attention, cross-attention to encoder outputs, and feed-forward layers). In our two-stage design, queries are first conditioned on the current entry and then refined using the log history, followed by a li… view at source ↗
Figure 3
Figure 3. Figure 3: Stage 1: Training only Q-Former adapter module. Text Encoder Q-former LLM Tokenize/Embedding Current Log Entry Reconstruction … … 2) Full Fine-Tuning Decoder-only Language model Learned Queries Embeddings Log History Current Log Entry [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Stage 2: Full fine-tuning of all modules. 4 Evaluation 4.1 Experimental Setup Datasets and preprocessing settings. To evaluate the ro￾bustness of CAPTAIN with respect to dataset characteristics, we conduct two experiments using datasets produced by dif￾ferent preprocessing pipelines. • Experiment 1 (AIRTAG-preprocessed ATLAS). We use the dataset preprocessed by the AIRTAG/ATLAS pipeline. • Experiment 2 (CA… view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study of Q-former and LM. We analyze the impact of (a) removing Q-Former (vanilla LM baselines with additional fine-tuning), (b) varying the history context window size, (c) replacing Q-Former with alternative adapters, and (d) swapping the decoder LM backbone. follows the order: Qwen3-0.6B (0.900) > Llama3.2-1B￾IT(0.899) > Gemma3-1B-IT (0.887). Qwen3-0.6B and Llama3.2-1B-IT achieve almost compara… view at source ↗
Figure 6
Figure 6. Figure 6: Wiener Filter Ablation. 5 Discussion 5.1 Robustness The results from Experiment 1 and Experiment 2 suggest a clear difference in robustness between AIRTAG and CAP￾TAIN. AIRTAG is sensitive to both the input token budget and the preprocessing pipeline, while CAPTAIN remains stable under these changes. We attribute this difference primarily to how each method represents log entries and performs anomaly detec… view at source ↗
Figure 7
Figure 7. Figure 7: Threshold–TPR/FPR relationship. C Unique log entries on S2 dataset During evaluation, we observed a notable performance drop on the S2 split. To understand this behavior, we inspected the logs in S2 and identified at least three types of benign log entries that are heavily skewed toward S2 and consistently yield high perplexity [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Sample Log Entry with High Perplexity Audit Success 9/15/2018 8:38:54 AM Microsoft￾Windows-Security-Auditing 4674 Sensitive Privilege Use "An operation was attempted on a privileged object. Subject: Security ID: WIN-D65GVM5K5FO\ aalsahee Account Name: aalsahee Account Domain: WIN-D65GVM5K5FO Logon ID: 0x19224 Object:^M Object Server: Security Object Type: File Object Name: C:\$Recycle.Bin\S -1-5-21-4500802… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 4 linked inside Pith

  1. [1]

    On the reproducibility of provenance-based intrusion detection that uses deep learning

    Talha Abrar, Ahmad Shamail, Mohammad Jaffer Iqbal, Amaan Ahmed, Muhammad Abdullah, Muhammad Shayan, Fareed Zaffar, Thomas Pasquier, David Ey- ers, and Ashish Gehani. On the reproducibility of provenance-based intrusion detection that uses deep learning. InProceedings of the 3rd ACM Conference on Reproducibility and Replicability, ACM REP ’25, page 14–28, ...

    2025

  2. [2]

    Clouseau: A hierarchical multi-agent approach for au- tonomous attack investigation

    Abdullah Aldaihan, Fahad Alotaibi, and Sergio Maffeis. Clouseau: A hierarchical multi-agent approach for au- tonomous attack investigation. InAnnual Computer Security Applications Conference, 2025

    2025

  3. [3]

    Berkay Celik, Xiangyu Zhang, and Dongyan Xu

    Abdulellah Alsaheel, Yuhong Nan, Shiqing Ma, Le Yu, Gregory Walkup, Z. Berkay Celik, Xiangyu Zhang, and Dongyan Xu. ATLAS: A sequence-based learning approach for attack investigation. In 30th USENIX Security Symposium (USENIX Secu- rity 21), pages 3005–3022. USENIX Association, Au- gust 2021. https://www.usenix.org/conference/ usenixsecurity21/presentatio...

    2021

  4. [4]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...

    1901

  5. [5]

    Zhuangbin Chen, Jinyang Liu, Wenwei Gu, Yuxin Su, and Michael R. Lyu. Experience report: Deep learning- based system log analysis for anomaly detection, 2022

    2022

  6. [6]

    Kairos: Practical intrusion detection and investigation using whole-system provenance

    Zijun Cheng, Qiujian Lv, Jinyuan Liang, Yan Wang, De- gang Sun, Thomas Pasquier, and Xueyuan Han. Kairos: Practical intrusion detection and investigation using whole-system provenance. In2024 IEEE Symposium on Security and Privacy (SP), pages 3533–3551, 2024

    2024

  7. [7]

    Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks

    Hyunjin Choi, Judong Kim, Seongho Joe, and Youngjune Gwon. Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks . In2020 25th International Conference on Pattern Recognition (ICPR), pages 5482–5487, Los Alamitos, CA, USA, January 2021. IEEE Computer Society

    2021

  8. [8]

    Clue: A high-performance, efficient, and robust apt detection framework via fine- tuning pretrained transformer and contrastive learning

    Wenzhuo Cui, Maihao Guo, Jingjing Feng, Shuyi Zhang, Zheng Liu, and Yu Wen. Clue: A high-performance, efficient, and robust apt detection framework via fine- tuning pretrained transformer and contrastive learning. In2025 International Conference on Intelligent Com- puting, 2025

    2025

  9. [9]

    gemma-3-1b-it (gemma 3 1b instruction-tuned) model card

    Google DeepMind. gemma-3-1b-it (gemma 3 1b instruction-tuned) model card. https://huggingface. co/google/gemma-3-1b-it. Accessed: 2026-01-29

    2026

  10. [10]

    BERT: pre-training of deep bidirec- tional transformers for language understanding.CoRR, abs/1810.04805, 2018

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirec- tional transformers for language understanding.CoRR, abs/1810.04805, 2018

    Pith/arXiv arXiv 2018

  11. [11]

    AIRTAG: Towards automated attack investigation by unsupervised learning with log texts

    Hailun Ding, Juan Zhai, Yuhong Nan, and Shiqing Ma. AIRTAG: Towards automated attack investigation by unsupervised learning with log texts. In32nd USENIX Security Symposium (USENIX Security 23), pages 373–390, Anaheim, CA, August 2023. USENIX Association. https://www.usenix.org/ conference/usenixsecurity23/presentation/ ding-hailun-airtag

    2023

  12. [12]

    Deeplog: Anomaly detection and diagnosis from system logs through deep learning

    Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communi- cations Security, CCS ’17, page 1285–1298, New York, NY , USA, 2017. Association for Computing Machinery

    2017

  13. [13]

    The llama 3 herd of models, 2024

    Abhimanyu Dubey et al. The llama 3 herd of models, 2024

    2024

  14. [14]

    Back- Propagating system dependency impact for attack investigation

    Pengcheng Fang, Peng Gao, Changlin Liu, Erman Ay- day, Kangkook Jee, Ting Wang, Yanfang (Fanny) Ye, Zhuotao Liu, and Xusheng Xiao. Back- Propagating system dependency impact for attack investigation. In31st USENIX Security Sympo- sium (USENIX Security 22), pages 2461–2478, Boston, MA, August 2022. USENIX Associa- tion. https://www.usenix.org/conference/ u...

    2022

  15. [15]

    Logbert: Log anomaly detection via bert

    Haixuan Guo, Shuhan Yuan, and Xintao Wu. Logbert: Log anomaly detection via bert. In2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2021

    2021

  16. [16]

    Log- gpt: Log anomaly detection via gpt

    Xiao Han, Shuhan Yuan, and Mohamed Trabelsi. Log- gpt: Log anomaly detection via gpt. In2023 IEEE International Conference on Big Data (BigData), pages 1117–1122, 2023. 15

    2023

  17. [17]

    Application of large language models in cybersecurity: A systematic literature review.IEEE Access, 12:176751–176778, 2024

    Ismayil Hasanov, Seppo Virtanen, Antti Hakkala, and Jouni Isoaho. Application of large language models in cybersecurity: A systematic literature review.IEEE Access, 12:176751–176778, 2024

    2024

  18. [18]

    A compre- hensive overview of large language models (llms) for cyber defences: Opportunities and directions, 2024

    Mohammed Hassanin and Nour Moustafa. A compre- hensive overview of large language models (llms) for cyber defences: Opportunities and directions, 2024

    2024

  19. [19]

    Pinjia He, Jieming Zhu, Shilin He, Jian Li, and Michael R. Lyu. Towards automated log parsing for large-scale log data analysis.IEEE Transactions on De- pendable and Secure Computing, 15(6):931–944, 2018

    2018

  20. [20]

    Ml-mamba: Efficient multi-modal large language model utilizing mamba-2, 2024

    Wenjun Huang, Jiakai Pan, Jiahao Tang, Yanyu Ding, Yifei Xing, Yuhe Wang, Zhengzhuo Wang, and Jianguo Hu. Ml-mamba: Efficient multi-modal large language model utilizing mamba-2, 2024

    2024

  21. [21]

    MAGIC: Detect- ing advanced persistent threats via masked graph representation learning

    Zian Jia, Yun Xiong, Yuhong Nan, Yao Zhang, Jinjing Zhao, and Mi Wen. MAGIC: Detect- ing advanced persistent threats via masked graph representation learning. In33rd USENIX Secu- rity Symposium (USENIX Security 24), pages 5197– 5214, Philadelphia, PA, August 2024. USENIX As- sociation. https://www.usenix.org/conference/ usenixsecurity24/presentation/jia-zian

    2024

  22. [22]

    Is bert really robust? a strong baseline for natural language attack on text classification and entail- ment

    Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entail- ment. InAAAI Conference on Artificial Intelligence, 2019

    2019

  23. [23]

    Deep learning for anomaly de- tection in log data: A survey.Machine Learning with Applications, 12:100470, June 2023

    Max Landauer, Sebastian Onder, Florian Skopik, and Markus Wurzenberger. Deep learning for anomaly de- tection in log data: A survey.Machine Learning with Applications, 12:100470, June 2023

    2023

  24. [24]

    Lanobert: System log anomaly detection based on bert masked language model.Appl

    Yukyung Lee, Jina Kim, and Pilsung Kang. Lanobert: System log anomaly detection based on bert masked language model.Appl. Soft Comput., 146(C), October 2023

    2023

  25. [25]

    BART: De- noising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: De- noising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors,Proceedings of the 58th Annual Mee...

    2020

  26. [26]

    Conlbs: An attack investigation approach using contrastive learning with behavior sequence.Sensors, 23(24), 2023

    Jiawei Li, Ru Zhang, and Jianyi Liu. Conlbs: An attack investigation approach using contrastive learning with behavior sequence.Sensors, 23(24), 2023

    2023

  27. [27]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    2023

  28. [28]

    Failure prediction in ibm bluegene/l event logs

    Yinglung Liang, Yanyong Zhang, Hui Xiong, and Ra- mendra Sahoo. Failure prediction in ibm bluegene/l event logs. InSeventh IEEE International Conference on Data Mining (ICDM 2007), pages 583–588, 2007

    2007

  29. [29]

    Log clustering based problem identification for online service systems

    Qingwei Lin, Hongyu Zhang, Jian-Guang Lou, Yu Zhang, and Xuewei Chen. Log clustering based problem identification for online service systems. In 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pages 102–111, 2016

    2016

  30. [30]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26286–26296, 2024

    2024

  31. [31]

    Mining invariants from console logs for system problem detection

    Jian-Guang Lou, Qiang Fu, Shengqi Yang, Ye Xu, and Jiang Li. Mining invariants from console logs for system problem detection. InProceedings of the 2010 USENIX Conference on USENIX Annual Technical Conference, USENIXATC’10, page 24, USA, 2010. USENIX Asso- ciation

    2010

  32. [32]

    Trec: Apt tactic / technique recognition via few-shot provenance subgraph learning

    Mingqi Lv, Hongzhe Gao, Xuebo Qiu, Tieming Chen, Tiantian Zhu, Jinyin Chen, and Shouling Ji. Trec: Apt tactic / technique recognition via few-shot provenance subgraph learning. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS ’24, page 139–152, New York, NY , USA,

    2024

  33. [33]

    Knowhow: Automatically applying high-level cti knowledge for interpretable and accurate provenance analysis, 2025

    Yuhan Meng, Shaofei Li, Jiaping Gui, Peng Jiang, and Ding Li. Knowhow: Automatically applying high-level cti knowledge for interpretable and accurate provenance analysis, 2025

    2025

  34. [34]

    Llama 3.2 1b instruct, 2024

    Meta. Llama 3.2 1b instruct, 2024. Model card and weights release. Release date: Sept 25, 2024. https://huggingface.co/meta-llama/Llama-3. 2-1B-Instruct

    2024

  35. [35]

    Training language models to follow instructions with human feedback.Advances in neural information pro- cessing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information pro- cessing systems, 35:27730–27744, 2022

    2022

  36. [36]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.J. Mach. Learn. Res., 21(1), January 2020. 16

    2020

  37. [37]

    Sentence-bert: Sen- tence embeddings using siamese bert-networks.ArXiv, abs/1908.10084, 2019

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sen- tence embeddings using siamese bert-networks.ArXiv, abs/1908.10084, 2019

    Pith/arXiv arXiv 1908

  38. [38]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.ArXiv, abs/1910.01108, 2019

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.ArXiv, abs/1910.01108, 2019

    Pith/arXiv arXiv 1910

  39. [39]

    GLU variants improve transformer

    Noam Shazeer. GLU variants improve transformer. CoRR, abs/2002.05202, 2020

    Pith/arXiv arXiv 2002

  40. [40]

    Anomaly detection for web log data analysis: A review.Journal of Alge- braic Statistics, 13(1), 2022

    Meena Siwach and Suman Mann. Anomaly detection for web log data analysis: A review.Journal of Alge- braic Statistics, 13(1), 2022

    2022

  41. [41]

    From alerts to intelligence: A novel llm-aided framework for host-based intrusion detection, 2025

    Danyu Sun, Jinghuai Zhang, Jiacen Xu, Yu Zheng, Yuan Tian, and Zhou Li. From alerts to intelligence: A novel llm-aided framework for host-based intrusion detection, 2025

    2025

  42. [42]

    Gemma 3 technical report, 2025

    Gemma Team et al. Gemma 3 technical report, 2025

    2025

  43. [43]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017

    2017

  44. [44]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xix- uan Song, Jiazheng Xu, Keqin Chen, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. Cogvlm: Visual expert for pretrained language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in...

    2024

  45. [45]

    Logppo: A log-based anomaly detector aided with prox- imal policy optimization algorithms.Smart Cities, 9(1), 2026

    Zhihao Wang, Jiachen Dong, and Chuanchuan Yang. Logppo: A log-based anomaly detector aided with prox- imal policy optimization algorithms.Smart Cities, 9(1), 2026

    2026

  46. [46]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of the 36th International Conference on Neural Infor- mation Processing Systems, NIPS ’22, Red Hook, NY , USA, 2022. Curran Associates Inc

    2022

  47. [47]

    Large language models for cyber security: A systematic literature review.ACM Trans

    Hanxiang Xu, Shenao Wang, Ningke Li, Kailong Wang, Yanjie Zhao, Kai Chen, Ting Yu, Yang Liu, and Haoyu Wang. Large language models for cyber security: A systematic literature review.ACM Trans. Softw. Eng. Methodol., September 2025. Just Accepted

    2025

  48. [48]

    Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael I. Jordan. Detecting large-scale system problems by mining console logs. InProceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP ’09, page 117–132, New York, NY , USA, 2009. Association for Computing Machinery

    2009

  49. [49]

    Deep learning-based intrusion detection systems: A survey, 2025

    Zhiwei Xu, Yujuan Wu, Shiheng Wang, Jiabao Gao, Tian Qiu, Ziqi Wang, Hai Wan, and Xibin Zhao. Deep learning-based intrusion detection systems: A survey, 2025

    2025

  50. [50]

    A survey on log anomaly detection using deep learning

    Rakesh Bahadur Yadav, P Santosh Kumar, and Sunita Vikrant Dhavale. A survey on log anomaly detection using deep learning. In2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), pages 1215–1220, 2020

    2020

  51. [51]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chen- gen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jian- wei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Ya...

    2025

  52. [52]

    +/-”. To assess the impact of this artifact, we conducted additional experiments where we removed the “+/-

    Xu Zhang, Yong Xu, Qingwei Lin, Bo Qiao, Hongyu Zhang, Yingnong Dang, Chunyu Xie, Xinsheng Yang, Qian Cheng, Ze Li, Junjie Chen, Xiaoting He, Randolph Yao, Jian-Guang Lou, Murali Chintalapati, Furao Shen, and Dongmei Zhang. Robust log-based anomaly de- tection on unstable log data. InProceedings of the 2019 27th ACM Joint Meeting on European Software Engi...

    2019

  53. [2024]

    Association for Computing Machinery