Abstract

The high rate of growth of advanced and highly obfuscated malware has made the use of conventional signature detection mechanisms less viable. In order to overcome this problem, this paper suggests a hybrid framework for malware classification based on the combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks to learn spatial and temporal features simultaneously. The given model converts malware feature vectors into structured grayscale representations from which CNN layers extract spatial features, and sequential opcode-related patterns are learned by an LSTM branch. Aspects of both domains are combined into a single embedding and used to classify nine malware families. Tests that have been carried out with a benchmark dataset show that the hybrid CNN-LSTM model attains a classification accuracy of 97.27% which is very high compared to baseline LSTM-only and CNN-only frameworks. The model demonstrates strong generalization, with weighted precision, recall, and F1-score values above 97% and highly discriminative ROC-AUC scores (reaching 1.000 for major classes). An in-depth analysis, such as confusion matrix analysis, precision-recall curves, and a comparison with the state-of-the-art methodologies demonstrate that the proposed architecture achieves performance comparable to that of leading malware detection models reported in recent literature. These findings validate the power of hybrid feature fusion to capture both static and dynamic behavioral traits of malware to provide a powerful, scalable, and highly accurate solution for next-generation cybersecurity systems.

Keywords

Malware Classification, CNN–LSTM Hybrid Model, Microsoft Malware Classification Challenge (BIG 2015), Deep Learning (DL), Cybersecurity, Opcode Sequences, Grayscale Image Features,

Downloads

Download data is not yet available.

References

  1. P. Thakur, V. Kansal, V. Rishiwal, Hybrid deep learning approach based on LSTM and CNN for malware detection. Wireless Personal Communications, 136, (2024) 1879–1901. https://doi.org/10.1007/s11277-024-11366-y
  2. M.A. Abdullah, Y. Yu, J. Cai, D. Addo, E.K. Bankas, Y.H. Gu, M.A. Al-antari, Deep learning IoT malware analysis: Investigation and understanding. Neural Computing and Applications, (2025) 1–28. https://doi.org/10.1007/s00521-025-11365-5
  3. J. Alotaibi, A hybrid software-defined networking approach for enhancing IoT cybersecurity with deep learning and blockchain in smart cities. Peer-to-Peer Networking and Applications, 18(3), (2025) 123. https://doi.org/10.1007/s12083-025-01935-8
  4. A.R.W. Sait, Explainable Deep Learning-Based Internet of Things Malware Detection Model. In: Kahraman, C., et al. Intelligent and Fuzzy Systems. INFUS 2025. Lecture Notes in Networks and Systems, Springer, Cham, 1529 (2025) 62–69. https://doi.org/10.1007/978-3-031-97992-7_8
  5. Y. Ye, L. Chen, S. Hou, W. Hardy, X. Li, DeepAM: A heterogeneous deep learning framework for intelligent malware detection. Knowledge and Information Systems, 54, (2018) 265–285. https://doi.org/10.1007/s10115-017-1058-9
  6. R. Chaganti, V. Ravi, T. D. Pham, Deep learning based cross architecture internet of things malware detection and classification. Computers & Security, 120, (2022) 102779. https://doi.org/10.1016/j.cose.2022.102779
  7. S. Hou, A. Saas, L. Chen, Y. Ye, (2016) Deep4maldroid: A deep learning framework for android malware detection based on linux kernel system call graphs. In 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops (WIW), IEEE, Omaha, NE, USA. https://doi.org/10.1109/WIW.2016.040
  8. W. Huang, J.W. Stokes, MtNet: A Multi-Task Neural Network for Dynamic Malware Classification. In: Caballero, J., Zurutuza, U., Rodríguez, R. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2016. Lecture Notes in Computer Science (), Springer, Cham, 9721. (2016) 399–418. https://doi.org/10.1007/978-3-319-40667-1_20
  9. T. Kim, B. Kang, M. Rho, S. Sezer, E.G. Im, A multimodal deep learning method for Android malware detection using various features. IEEE Transactions on Information Forensics and Security, 14, (2018) 773–788. https://doi.org/10.1109/TIFS.2018.2866319
  10. B. Kolosnjaji, A. Zarras, G. Webster, C. Eckert, Deep learning for classification of malware system call sequences. In Australasian Joint Conference on Artificial Intelligence, (2016) 137–149. https://doi.org/10.1007/978-3-319-50127-7_11
  11. R. Kodamanchili, M.L.S.N. S. Lakshmi, T.S. Tadivaka, V.N.S.R. Murthy, P.K.V. Kothapalli, V. S. Dhullipalla, Training neural networks for multi-class classification using softmax activation and cross-entropy loss: A comparative study against traditional machine learning models. In Proceeding 2026 International Conference on Emerging Research in Smart Electronics and Machine Informatics (ECMI), IEEE, Chikkamagaluru, India https://doi.org/10.1109/ECMI68341.2026.11602907
  12. M. Alshoulie, A. Mehmood, Deep learning approaches for malware detection: A comprehensive review. IEEE Access, 13, (2025) 118652 – 118677. https://doi.org/10.1109/ACCESS.2025.3582875
  13. Y. Li, L. Sun, Q. Yan, Z. Li, W. Srisa-An, H. Ye, Significant permission identification for machine-learning-based Android malware detection. IEEE Transactions on Industrial Informatics, 14(7), (2018) 3216–3225. https://doi.org/10.1109/TII.2017.2789219
  14. K.M. Sujon, R.B. Hassan, M. Abdullah-Al-Wadud, J. Uddin, OPTISTACK: A hybrid ensemble learning and XAI-based approach for malware detection. IEEE Access, 13, (2025) 104992 – 105026. https://doi.org/10.1109/ACCESS.2025.3579880
  15. L. Nataraj, S. Karthikeyan, G. Jacob, B.S. Manjunath, Malware images: visualization and automatic classification. In Proceedings of the 8th international symposium on visualization for cyber security, 4, (2011) 1-7. https://doi.org/10.1145/2016904.2016908
  16. R. Pascanu, J.W. Stokes, H. Sanossian, M. Marinescu, A. Thomas, (2015) Malware classification with recurrent networks. in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, South Brisbane, QLD, Australia. https://doi.org/10.1109/ICASSP.2015.7178304
  17. I.A. Mahar, K. Aziz, P. Chakrabarti, N. Ahmed, M. Ladan, Y. Javed, A hybrid machine learning approach for detecting DDoS attacks in software-defined networks. Scientific Reports, 16(1), (2026) 6533. https://doi.org/10.1038/s41598-026-35458-w
  18. M. Rhode, P. Burnap, K. Jones, Early-stage malware prediction using recurrent neural networks. Computers & Security, 77, (2018) 578–594. https://doi.org/10.1016/j.cose.2018.04.007
  19. P.V. Shijo, A.J.P.C.S. Salim, Integrated static and dynamic analysis for malware detection. Procedia Computer Science, 46, (2015) 804–811. https://doi.org/10.1016/j.procs.2015.02.149
  20. J. Saxe, K. Berlin, (2015) Deep neural network based malware detection using two-dimensional binary program features. In 2015 10th International Conference on Malicious and Unwanted Software (MALWARE), IEEE, Fajardo, PR, USA. https://doi.org/10.1109/MALWARE.2015.7413680
  21. A.A. Almazroi, N. Ayub, Deep learning hybridization for improved malware detection in smart Internet of Things. Scientific Reports, 14, (2024) 7838. https://doi.org/10.1038/s41598-024-57864-8
  22. C.W. Chen, S.P. Tseng, T.W. Kuan, J.F. Wang, Outpatient text classification using attention-based bidirectional LSTM. Information, 11(2), (2020) 106. https://doi.org/10.3390/info11020106
  23. Z. Yuan, Y. Lu, Y. Xue, Droiddetector: Android malware characterization and detection using deep learning. Tsinghua Science and Technology, 21, (2016) 114–123. https://doi.org/10.1109/TST.2016.7399288
  24. Y. Zhao, W. Cui, S. Geng, B. Bo, Y. Feng, W. Zhang, A malware detection method of code texture visualization based on an improved Faster RCNN combining transfer learning. IEEE Access, 8, (2020) 166630–166641. https://doi.org/10.1109/ACCESS.2020.3022722