Can Small GenAI Language Models Rival Large Language Models in Understanding Application Behavior?

Ali A. Ghorbani; Hamed Jelodar; Mohammad Meymani; Parisa Hamedi; Roozbeh Razavi-Far

REVIEW 4 major objections 5 minor 24 references

Small language models can match large ones on malware detection when asked directly, not through a classifier head.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 22:00 UTC pith:YGVAQ5AS

load-bearing objection A routine five-model malware-detection benchmark whose own Table III contradicts its headline; the small-model result is plausible but the evidence is too thin to carry it. the 4 major comments →

arxiv 2511.12576 v2 pith:YGVAQ5AS submitted 2025-11-16 cs.SE

Can Small GenAI Language Models Rival Large Language Models in Understanding Application Behavior?

Mohammad Meymani , Hamed Jelodar , Parisa Hamedi , Roozbeh Razavi-Far , Ali A. Ghorbani This is my paper

classification cs.SE

keywords small language modelslarge language modelsmalware detectionapplication behavior analysiszero-shot promptingbenchmark comparisonresource efficiencycode classification

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether small generative AI language models can stand in for large ones on a concrete software-security task: deciding whether a piece of source code is malware. On a 10,000-code sample of benign and malicious programs, it compares five models, from a 1.3-billion-parameter coder to 7-8-billion-parameter generalists, using both a frozen classification head and direct zero-shot prompting. The central finding is that a small instruction-tuned model, Phi-4-mini, reaches the highest malware-class F1 score (87%) under prompting, while the larger models land between 79% and 84%. The paper argues this makes small models a resource-efficient complement to large ones: they run faster and cheaper while staying competitive on the classes that matter for malware detection.

Core claim

Using a 10,000-sample subset (5,000 benign, 5,000 malware) drawn from the SBAN code benchmark, the authors evaluate DeepSeek-1.3B, Phi-4-mini, Llama-3.1-8B, Qwen2.5-7B, and Mistral-7B under two protocols. Classification heads attached to frozen embeddings produce near-chance or badly skewed results, so the authors set that protocol aside. Under zero-shot prompting with a strict one-word answer and keyword-based extraction, the small Phi-4-mini attains 86% accuracy and 87% F1 on the malware class, ahead of Qwen2.5-7B (85%/83%), Llama-3.1-8B (82%/84%), DeepSeek-1.3B (54%/67%), and Mistral-7B (82%/79%). The paper reads this as evidence that small models can rival large ones on malware detection

What carries the argument

Zero-shot prompting: each model is asked to analyze a code snippet and reply with a single word, 'malware' or 'benign', and the answer is extracted by trimming and keyword search. This prompt protocol, rather than the frozen classification head (a sigmoid over logits with a 0.5 threshold), is the mechanism that lets small models show competitive detection. The contrast between the two protocols is itself the paper's main experimental lever: the head fails across the board, while prompting succeeds, shifting the comparison from model size to instruction-following ability.

Load-bearing premise

The SBAN benchmark and the un-described 10,000-sample subset fairly represent the task of understanding application behavior for malware detection; if the benign/malware labels or the sample selection are skewed, every model ranking in the paper inherits that skew.

What would settle it

Re-run the same five models with the same zero-shot prompt on an independently labeled malware/benign source-code corpus whose sampling is fully disclosed. If Phi-4-mini no longer matches or exceeds the larger models on malware-class F1, and no other small model comes close, the paper's central claim that small models can complement large ones loses support. A cheaper secondary check: re-run Phi-4-mini's answers through constrained decoding (allow only 'malware' or 'benign') and see whether its 87% F1 changes materially.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Security tools that cannot afford a large model on edge hardware can deploy a small prompted model and still catch malware at a competitive rate.
Model selection for code-based malware triage should be based on class-level F1 and inference cost, not headline accuracy alone.
Prompting an off-the-shelf model can yield competitive detection without task-specific fine-tuning, lowering deployment complexity.
Larger models remain the safer default only when maximizing overall accuracy, rather than malware-class F1, is the sole requirement.
Inference time scales with model size, so the small models' speed advantage translates directly into operational throughput for real-time screening.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If small-model competence on code classification generalizes beyond malware to vulnerability detection or code summarization, the resource-efficiency argument could reshape how static-analysis pipelines are built for edge devices.
The near-random classification-head results suggest frozen embeddings do not separate benign from malicious code; a cheaply fine-tuned small model might close the accuracy gap with large models even further, an experiment the paper lists as future work.
The keyword-extraction step in the prompt protocol is a hidden variable: stricter output parsing (for example, constrained decoding that only allows 'malware' or 'benign') could raise or lower each model's measured F1, so the reported ranking depends partly on the parsing rule, not just model capability.
Because the benchmark and sampling procedure are not externally validated, an independent replication on a differently labeled code corpus would test whether the small-model advantage is a property of the task or of this particular data split.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A routine five-model malware-detection benchmark whose own Table III contradicts its headline; the small-model result is plausible but the evidence is too thin to carry it.

read the letter

You asked about arXiv:2511.12576. The short version: this is an empirical comparison of five off-the-shelf LLM/SLM checkpoints on a 10k-sample subset of SBAN, a dataset built by the same group. The reader's REJECT is about right, but the paper isn't garbage — it's honest arithmetic on one internal benchmark, and the claim that a well-prompted small model can keep pace with larger ones is plausible. The problem is the paper doesn't support its own headline.

What's actually new: not much as a result type. The paper itself cites earlier LLM malware detection work (refs 10, 11) and an SLM-vs-LLM comparison (ref 16). What is new is the specific set of five checkpoints and the prompt-based result showing Phi-4-mini at 86% accuracy and 87% F1 for malware, ahead of Qwen (85%) and Llama (82%). That's a concrete measurement, and the arithmetic of Table III checks out — F1 and accuracy reconcile with precision/recall, so the numbers look computed honestly. Credit is also due for reporting both classification-head and prompt-based results, even though only one ends up used.

Soft spots, in order of weight. First, the internal contradiction: the abstract and discussion say 'larger models generally achieve higher overall accuracy,' but Table III's prompt-based column has the smallest model, Phi-4-mini, at 86% accuracy, the best of the five. The text's own bullet list says Qwen has the highest overall accuracy, which is only true if you ignore the malware class or average oddly. This is not a peripheral flaw — it's the headline claim contradicting the data. Second, the classification-head experiments, listed as a contribution, are dropped after producing 'unexpected results.' That's post-hoc exclusion of half the study. Third, the efficiency claims (faster inference, lower cost) are asserted but never measured — the only timing mention is a vague 'Qwen-7B and Mistral-7B require roughly 11 minutes for 1000 samples,' no table, no variance. Fourth, the benchmark is self-authored (SBAN), the 10k subset sampling is not described, and no external validation or release is given. All comparisons inherit any bias in that dataset. Finally, single runs, no error bars.

The stress-test note about the benchmark is on target. Because the subset is not released and the sampling is unstated, the numbers only describe an unverifiable internal dataset. That said, the conclusion that small models can complement large ones is not absurd — it's consistent with other published results. What's missing is the rigor to make it a claim rather than a hypothesis.

Who is this for? Practitioners wanting a quick orientation of specific HuggingFace checkpoints on a large code dataset might skim it, but it's not a reliable citation for the SLM-vs-LLM claim. The paper deserves a serious referee because the question is real and the experimental setup is reproducible in principle — but it needs major revision: fix the internal contradiction, report the classification-head results honestly or justify excluding them, add timing measurements, describe the subset, release the data and code, and run multiple seeds. If it were resubmitted with those fixes, it could be a solid small contribution. As it stands, the evidence doesn't carry the headline.

Recommendation: engage with the work in the sense of checking whether the authors address the contradiction and provide the missing measurements; but I would not cite it as evidence for the SLM-competitiveness claim until it is revised.

Referee Report

4 major / 5 minor

Summary. The paper evaluates five GenAI language models (DeepSeek-1.3B, Phi-4-mini, Llama-3.1-8B, Qwen-2.5-7B, Mistral-7B) on binary malware detection using 10,000 samples from the authors' SBAN dataset. Two strategies are compared: a classification head over pretrained model outputs and zero-shot prompting with keyword-based answer extraction. Metrics (accuracy, precision, recall, F1) are reported per class for both strategies. The paper concludes that smaller models, particularly Phi-4-mini, can rival larger models while providing better resource efficiency.

Significance. If the empirical comparison were trustworthy, the paper would be a useful practical benchmark for SLM/LLM trade-offs in malware detection. The manuscript has some strengths: a balanced 5,000/5,000 evaluation set, per-class metrics, a direct comparison of classification-head and prompting strategies, and internally consistent arithmetic in Table III (F1 and accuracy reconcile with precision/recall on the balanced classes). However, the evaluation rests on a self-created, unreleased benchmark with no described sampling or label validation, and the reported differences are not supported by uncertainty quantification. These issues, combined with direct contradictions between the abstract and Table III, mean that the central claim is currently unsupported.

major comments (4)

[Abstract and Table III; §IV-C] The abstract's claim that 'larger models generally achieve higher overall accuracy' is contradicted by Table III: in the prompt-based method, the smallest model, Phi-4-mini, achieves 86% accuracy, above Qwen-2.5-7B (85%), Llama-3.1-8B (82%), and Mistral-7B (82%). The bullet in §IV-C stating 'Qwen2.5-Coder achieves the highest overall accuracy' is also false against the table. Since the paper's central message concerns small-versus-large performance, this inconsistency must be resolved with corrected reporting and a revised claim.
[Section III-A] The evaluation relies entirely on a 10,000-sample subset of SBAN [9], a dataset created by the authors' own group. The sampling procedure, seed, stratification, inclusion/exclusion criteria, and label validation are not described, and the dataset/subset is not released. Without external validation or a detailed protocol, all model comparisons in Table III are unverifiable and may reflect dataset-specific artifacts rather than general application-behavior understanding. The paper should either release the subset, provide full sampling details, and/or evaluate on an independent public benchmark.
[Section IV-A and IV-C] No uncertainty quantification is provided: the models are evaluated once ('single inference pass'), with no repeated runs, confidence intervals, or significance tests. For a balanced 10,000-sample evaluation, the 1-point difference between Phi-4-mini (86%) and Qwen-2.5-7B (85%) is within sampling error, so claims that one model 'rivals' or 'outperforms' another are not supported. Additionally, the inference-time advantages asserted in the abstract are not quantified anywhere in the paper.
[Section IV-C and V] The classification-head results are dismissed as 'unexpected results' and excluded from detailed analysis, but they are still reported in Table III and used in the comparison; the reason for the failure (e.g., untrained head, thresholding, tokenization) is never examined. Section V states that SLMs 'exhibit lower recall for benign,' yet Phi-4-mini (prompt-based) has 81% benign recall, higher than DeepSeek's 14%; this generalization is not supported by the table. The conclusion also calls Qwen-2.5-7B a 'smaller model' despite the paper's own grouping in Fig. 1.

minor comments (5)

[§IV-C, bullet 3] 'fail to detect benign is (class ’1’)' should read 'class 0'; the sentence is confusing as written.
[References] Reference [9] is missing a venue or identifier; references [10], [12], [14], and [15] appear as 'X. e. a. Author' instead of full author lists; citation [3, 3, 4] duplicates [3].
[Naming consistency] The model is called 'Qwen2.5-Coder' in §IV-C but 'Qwen-2.5-7B' in Table II; the conclusion groups Qwen-2.5-7B with 'smaller models' even though Fig. 1 classifies it as an LLM.
[Figure 1] The 'Yes No' labels in the workflow diagram are ambiguous, and the prompt box is not clearly connected to the answer-extraction step.
[Section IV-A] The classification-head evaluation is not fully described: no training procedure for the lightweight head is given, and the phrase 'pre-trained weights were employed for classification' does not explain how the head is obtained or thresholded.

Circularity Check

0 steps flagged

No significant circularity: the paper is an empirical benchmark comparison; model scores are measured, not derived from or fitted to the conclusion.

full rationale

This is an empirical comparison study, not a derivational chain. The models' accuracy, precision, recall, and F1 numbers are obtained by running publicly available model checkpoints on a fixed 10,000-sample subset of the SBAN dataset, with no fitted parameters or closed-form predictions that could reduce to the inputs. No equation in the paper defines a result in terms of the conclusion, no prediction is a renamed fitted quantity, and no uniqueness theorem or prior-work result is used to force the choice of models or conclusions. The dataset itself (SBAN, Ref. [9]) is authored by the same research group and its sampling procedure is underspecified, which is a legitimate validity/benchmark-quality concern; however, it does not create a circularity because the model outputs are not constructed from the labels or from the paper's claims. Similarly, the tension between the abstract's statement that larger models generally achieve higher accuracy and Table III showing Phi-4-mini with the highest prompt-based accuracy is an internal-consistency issue, not a circularity issue. The reported findings are therefore self-contained as an empirical measurement, even though their external validity depends on the unverified benchmark.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper's results rest almost entirely on a self-authored benchmark (SBAN, Ref [9]) with no external validation, a hand-chosen 10,000-sample subset, and standard hyperparameter defaults. No free parameters are fitted to make a derivation work; the study is purely an empirical comparison, so the ledger's burden is concentrated in the benchmark and operationalization assumptions. No new entities are introduced.

free parameters (3)

Classification threshold = 0.5
Default sigmoid threshold applied to logits in Eq. 1; chosen by hand rather than tuned. Affects the classification-head results that were later abandoned (Sec. III-C1).
Maximum sequence length = 1024 tokens
Chosen by hand (Sec. IV-A1); longer source files are truncated, which can bias results for either class.
Prompt temperature = 0
Chosen by hand for reproducibility of greedy decoding (Sec. IV-A2); standard but still a modeling choice.

axioms (4)

domain assumption SBAN dataset and its 10,000-sample subset provide valid ground-truth labels for malware vs benign.
Invoked in Sec. III-A; the subset sampling procedure is unspecified and SBAN [9] is authored by the same group with no external validation shown.
domain assumption Binary malware classification is an adequate operationalization of 'understanding application behavior'.
The title/abstract generalize from a two-class source-code task to general application-behavior understanding; Sec. I frames malware detection as 'a representative task.'
domain assumption Zero-shot prompting with greedy decoding and keyword extraction yields answers comparable across models.
Sec. III-C2 and IV-A2; no analysis of malformed outputs or extraction failures is reported.
standard math Standard binary classification metrics (accuracy, precision, recall, F1) computed on a balanced 5,000/5,000 split.
Eqs. (2)-(5) in Sec. III-D; arithmetic internal consistency checks out for Table III.

pith-pipeline@v1.3.0-alltime-deepseek · 6381 in / 21645 out tokens · 170233 ms · 2026-08-03T22:00:26.799157+00:00 · methodology

0 comments

read the original abstract

Generative AI (GenAI) models, particularly large language models (LLMs), have transformed multiple domains, including natural language processing, software analysis, and code understanding. Their ability to analyze and generate code has enabled applications such as source code summarization, behavior analysis, and malware detection. In this study, we systematically evaluate the capabilities of both small and large GenAI language models in understanding application behavior, with a particular focus on malware detection as a representative task. While larger models generally achieve higher overall accuracy, our experiments show that small GenAI models maintain competitive precision and recall, offering substantial advantages in computational efficiency, faster inference, and deployment in resource-constrained environments. We provide a detailed comparison across metrics such as accuracy, precision, recall, and F1-score, highlighting each model's strengths, limitations, and operational feasibility. Our findings demonstrate that small GenAI models can effectively complement large ones, providing a practical balance between performance and resource efficiency in real-world application behavior analysis.

Figures

Figures reproduced from arXiv: 2511.12576 by Ali A. Ghorbani, Hamed Jelodar, Mohammad Meymani, Parisa Hamedi, Roozbeh Razavi-Far.

**Figure 2.** Figure 2: Grouped bar chart comparing weighted average and macro average on Precision, Recall, and F1-score across both classes for each model. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: Line charts showing the variation of performance metrics. Each chart focuses on a single metric for clearer model performance comparison. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Heatmap representing Precision, Recall, and F1-score for both [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

24 extracted references · 9 linked inside Pith

[1]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

2017
[2]

Large language models (llms) for source code analysis: applications, models and datasets,

H. Jelodar, M. Meymani, and R. Razavi-Far, “Large language models (llms) for source code analysis: applications, models and datasets,” arXiv preprint arXiv:2503.17502, 2025

Pith/arXiv arXiv 2025
[3]

Scaling down to scale up: A cost-benefit analysis of replacing openai’s llm with open source slms in production,

C. Irugalbandara, A. Mahendra, R. Daynauth, T. K. Arachchige, J. Dantanarayana, K. Flautner, L. Tang, Y . Kang, and J. Mars, “Scaling down to scale up: A cost-benefit analysis of replacing openai’s llm with open source slms in production,” in2024 IEEE International Sym- posium on Performance Analysis of Systems and Software (ISPASS), pp. 280–291, IEEE, 2024

2024
[4]

A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness,

F. Wang, Z. Zhang, X. Zhang, Z. Wu, T. Mo, Q. Lu, W. Wang, R. Li, J. Xu, X. Tang,et al., “A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness,”ACM Trans- actions on Intelligent Systems and Technology, 2024

2024
[5]

A survey of llm- based agents: Theories, technologies, applications and suggestions,

X. Dong, X. Zhang, W. Bu, D. Zhang, and F. Cao, “A survey of llm- based agents: Theories, technologies, applications and suggestions,” in 2024 3rd International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC), pp. 407–413, IEEE, 2024

2024
[7]

Nld-llm: A systematic framework for evaluating small language transformer models on natural language description,

H. Jelodar, M. Meymani, P. Hamedi, T. E. Nwankwo, S. Bai, R. Razavi-Far, and A. A. Ghorbani, “Nld-llm: A systematic framework for evaluating small language transformer models on natural language description,”arXiv preprint arXiv:2510.05139, 2025

arXiv 2025
[8]

Xgen- q: An explainable domain-adaptive llm framework with retrieval- augmented generation for software security,

H. Jelodar, M. Meymani, R. Razavi-Far, and A. A. Ghorbani, “Xgen- q: An explainable domain-adaptive llm framework with retrieval- augmented generation for software security,” 2025

2025
[9]

Sban: A framework & multi-dimensional dataset for large language model pre-training and software code mining,

H. Jelodar, M. Meymani, S. Bai, R. Razavi-Far, and A. A. Ghorbani, “Sban: A framework & multi-dimensional dataset for large language model pre-training and software code mining,” 2025

2025
[10]

Malicious code detection using large language models: A survey,

A.-K. e. a. Hossain, “Malicious code detection using large language models: A survey,”arXiv preprint arXiv:2409.07587, 2024

Pith/arXiv arXiv 2024
[11]

Static malware analysis with llms,

R. Fujii and T. Yamagishi, “Static malware analysis with llms,”arXiv preprint arXiv:2411.14905, 2024

Pith/arXiv arXiv 2024
[12]

Llm-assisted malware variant generation and detec- tion,

X. e. a. Huang, “Llm-assisted malware variant generation and detec- tion,”arXiv preprint arXiv:2504.07137, 2025

Pith/arXiv arXiv 2025
[13]

Deep security challenge: Malware detection with llms,

Y . M. P. P. Kou, “Deep security challenge: Malware detection with llms,” 2024

2024
[14]

Llmalmorph: Leveraging llms to generate malware vari- ants,

e. a. Akil, “Llmalmorph: Leveraging llms to generate malware vari- ants,”arXiv preprint arXiv:2507.09411, 2025

arXiv 2025
[15]

Mitigating distribution shifts in graph-based an- droid malware classification with llm embeddings,

e. a. Tran, “Mitigating distribution shifts in graph-based an- droid malware classification with llm embeddings,”arXiv preprint arXiv:2508.06734, 2025

arXiv 2025
[16]

Small language models can outperform humans in short creative writing: A study comparing slms with humans and llms,

G. Marco, L. Rello, and J. Gonzalo, “Small language models can outperform humans in short creative writing: A study comparing slms with humans and llms,”arXiv preprint arXiv:2409.11547, 2024

Pith/arXiv arXiv 2024
[17]

Deepseek-v3 technical report,

A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan,et al., “Deepseek-v3 technical report,”arXiv preprint arXiv:2412.19437, 2024

Pith/arXiv arXiv 2024
[18]

Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras,

A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V . Chaudhary, C. Chen,et al., “Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras,”arXiv preprint arXiv:2503.01743, 2025

Pith/arXiv arXiv 2025
[19]

Llama: Open and efficient foundation language models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi`ere, N. Goyal, E. Hambro, F. Azhar,et al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

Pith/arXiv arXiv 2023
[20]

Qwen2. 5-coder technical report,

B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu,et al., “Qwen2. 5-coder technical report,”arXiv preprint arXiv:2409.12186, 2024

Pith/arXiv arXiv 2024
[21]

Mistral 7b,

A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” 2023

2023
[22]

Fewfine: Few-shot malware traffic classification via transfer learning based on fine-tuning strategy,

X. Liu, M. Shen, L. Cui, K. Ye, J. Jia, and G. Yue, “Fewfine: Few-shot malware traffic classification via transfer learning based on fine-tuning strategy,” in2022 IEEE Smartworld, Ubiquitous Intelli- gence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld/UIC/ScalCom/Dig...

2022
[23]

Exploring the application of transfer learning in malware detection by fine-tuning pre- trained models on binary classification to new datasets on multi-class classification,

B. Ajayi, B. Barakat, K. McGarry, and M. Abukeshek, “Exploring the application of transfer learning in malware detection by fine-tuning pre- trained models on binary classification to new datasets on multi-class classification,” in2024 29th International Conference on Automation and Computing (ICAC), pp. 1–6, IEEE, 2024

2024
[24]

Llm- maldetect: A large language model-based method for android malware detection,

R. Feng, H. Chen, S. Wang, M. M. Karim, and Q. Jiang, “Llm- maldetect: A large language model-based method for android malware detection,”IEEE Access, 2025

2025
[25]

Prompt chaining-assisted malware detection: A hybrid approach utilizing fine-tuned llms and domain knowledge-enriched cybersecurity knowledge graphs,

N. M. Kumar, F. T. Lisa, and S. R. Islam, “Prompt chaining-assisted malware detection: A hybrid approach utilizing fine-tuned llms and domain knowledge-enriched cybersecurity knowledge graphs,” in2024 IEEE International Conference on Big Data (BigData), pp. 1672–1677, IEEE, 2024

2024

[1] [1]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

2017

[2] [2]

Large language models (llms) for source code analysis: applications, models and datasets,

H. Jelodar, M. Meymani, and R. Razavi-Far, “Large language models (llms) for source code analysis: applications, models and datasets,” arXiv preprint arXiv:2503.17502, 2025

Pith/arXiv arXiv 2025

[3] [3]

Scaling down to scale up: A cost-benefit analysis of replacing openai’s llm with open source slms in production,

C. Irugalbandara, A. Mahendra, R. Daynauth, T. K. Arachchige, J. Dantanarayana, K. Flautner, L. Tang, Y . Kang, and J. Mars, “Scaling down to scale up: A cost-benefit analysis of replacing openai’s llm with open source slms in production,” in2024 IEEE International Sym- posium on Performance Analysis of Systems and Software (ISPASS), pp. 280–291, IEEE, 2024

2024

[4] [4]

A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness,

F. Wang, Z. Zhang, X. Zhang, Z. Wu, T. Mo, Q. Lu, W. Wang, R. Li, J. Xu, X. Tang,et al., “A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness,”ACM Trans- actions on Intelligent Systems and Technology, 2024

2024

[5] [5]

A survey of llm- based agents: Theories, technologies, applications and suggestions,

X. Dong, X. Zhang, W. Bu, D. Zhang, and F. Cao, “A survey of llm- based agents: Theories, technologies, applications and suggestions,” in 2024 3rd International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC), pp. 407–413, IEEE, 2024

2024

[6] [7]

Nld-llm: A systematic framework for evaluating small language transformer models on natural language description,

H. Jelodar, M. Meymani, P. Hamedi, T. E. Nwankwo, S. Bai, R. Razavi-Far, and A. A. Ghorbani, “Nld-llm: A systematic framework for evaluating small language transformer models on natural language description,”arXiv preprint arXiv:2510.05139, 2025

arXiv 2025

[7] [8]

Xgen- q: An explainable domain-adaptive llm framework with retrieval- augmented generation for software security,

H. Jelodar, M. Meymani, R. Razavi-Far, and A. A. Ghorbani, “Xgen- q: An explainable domain-adaptive llm framework with retrieval- augmented generation for software security,” 2025

2025

[8] [9]

Sban: A framework & multi-dimensional dataset for large language model pre-training and software code mining,

H. Jelodar, M. Meymani, S. Bai, R. Razavi-Far, and A. A. Ghorbani, “Sban: A framework & multi-dimensional dataset for large language model pre-training and software code mining,” 2025

2025

[9] [10]

Malicious code detection using large language models: A survey,

A.-K. e. a. Hossain, “Malicious code detection using large language models: A survey,”arXiv preprint arXiv:2409.07587, 2024

Pith/arXiv arXiv 2024

[10] [11]

Static malware analysis with llms,

R. Fujii and T. Yamagishi, “Static malware analysis with llms,”arXiv preprint arXiv:2411.14905, 2024

Pith/arXiv arXiv 2024

[11] [12]

Llm-assisted malware variant generation and detec- tion,

X. e. a. Huang, “Llm-assisted malware variant generation and detec- tion,”arXiv preprint arXiv:2504.07137, 2025

Pith/arXiv arXiv 2025

[12] [13]

Deep security challenge: Malware detection with llms,

Y . M. P. P. Kou, “Deep security challenge: Malware detection with llms,” 2024

2024

[13] [14]

Llmalmorph: Leveraging llms to generate malware vari- ants,

e. a. Akil, “Llmalmorph: Leveraging llms to generate malware vari- ants,”arXiv preprint arXiv:2507.09411, 2025

arXiv 2025

[14] [15]

Mitigating distribution shifts in graph-based an- droid malware classification with llm embeddings,

e. a. Tran, “Mitigating distribution shifts in graph-based an- droid malware classification with llm embeddings,”arXiv preprint arXiv:2508.06734, 2025

arXiv 2025

[15] [16]

Small language models can outperform humans in short creative writing: A study comparing slms with humans and llms,

G. Marco, L. Rello, and J. Gonzalo, “Small language models can outperform humans in short creative writing: A study comparing slms with humans and llms,”arXiv preprint arXiv:2409.11547, 2024

Pith/arXiv arXiv 2024

[16] [17]

Deepseek-v3 technical report,

A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan,et al., “Deepseek-v3 technical report,”arXiv preprint arXiv:2412.19437, 2024

Pith/arXiv arXiv 2024

[17] [18]

Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras,

A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V . Chaudhary, C. Chen,et al., “Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras,”arXiv preprint arXiv:2503.01743, 2025

Pith/arXiv arXiv 2025

[18] [19]

Llama: Open and efficient foundation language models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi`ere, N. Goyal, E. Hambro, F. Azhar,et al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

Pith/arXiv arXiv 2023

[19] [20]

Qwen2. 5-coder technical report,

B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu,et al., “Qwen2. 5-coder technical report,”arXiv preprint arXiv:2409.12186, 2024

Pith/arXiv arXiv 2024

[20] [21]

Mistral 7b,

A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” 2023

2023

[21] [22]

Fewfine: Few-shot malware traffic classification via transfer learning based on fine-tuning strategy,

X. Liu, M. Shen, L. Cui, K. Ye, J. Jia, and G. Yue, “Fewfine: Few-shot malware traffic classification via transfer learning based on fine-tuning strategy,” in2022 IEEE Smartworld, Ubiquitous Intelli- gence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld/UIC/ScalCom/Dig...

2022

[22] [23]

Exploring the application of transfer learning in malware detection by fine-tuning pre- trained models on binary classification to new datasets on multi-class classification,

B. Ajayi, B. Barakat, K. McGarry, and M. Abukeshek, “Exploring the application of transfer learning in malware detection by fine-tuning pre- trained models on binary classification to new datasets on multi-class classification,” in2024 29th International Conference on Automation and Computing (ICAC), pp. 1–6, IEEE, 2024

2024

[23] [24]

Llm- maldetect: A large language model-based method for android malware detection,

R. Feng, H. Chen, S. Wang, M. M. Karim, and Q. Jiang, “Llm- maldetect: A large language model-based method for android malware detection,”IEEE Access, 2025

2025

[24] [25]

Prompt chaining-assisted malware detection: A hybrid approach utilizing fine-tuned llms and domain knowledge-enriched cybersecurity knowledge graphs,

N. M. Kumar, F. T. Lisa, and S. R. Islam, “Prompt chaining-assisted malware detection: A hybrid approach utilizing fine-tuned llms and domain knowledge-enriched cybersecurity knowledge graphs,” in2024 IEEE International Conference on Big Data (BigData), pp. 1672–1677, IEEE, 2024

2024