MalVis: A Large-Scale Image-Based Framework and Dataset for Advancing Android Malware Classification

As technology advances, Android malware continues to pose significant threats to devices and sensitive data. The open-source nature of the Android OS and the availability of its SDK contribute to this rapid growth. Traditional malware detection techniques, such as signature-based, static, and dynamic analysis, struggle to detect obfuscated threats that use encryption, packing, or compression. While deep learning (DL)-based visualization methods have been proposed, they often fail to highlight the critical malicious features effectively. This research introduces MalVis, a unified visualization framework that integrates entropy and N-gram analysis to emphasize structural and anomalous patterns in malware bytecode. MalVis addresses key limitations of prior methods, including insufficient feature representation, poor interpretability, and limited data accessibility. The framework leverages a newly introduced large-scale dataset, the MalVis dataset, containing over 1.3 million visual samples across nine malware classes and one benign class. We evaluate MalVis against state-of-the-art visualization techniques using leading CNN models: MobileNet-V2, DenseNet201, ResNet50, and Inception-V3. To enhance performance and reduce overfitting, we implement eight ensemble learning strategies. Additionally, an undersampling technique mitigates class imbalance in the multiclass setting. MalVis achieves strong results: 95.19% accuracy, 90.81% F1-score, 92.58% precision, 89.10% recall, 87.58% MCC, and 98.06% ROC-AUC. These findings demonstrate the effectiveness of MalVis in enabling accurate, interpretable malware detection and providing a valuable resource for security research and applications.