An-Najah University Journal for Research - A (Natural Sciences)

Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach

Article info

2023-09-11
2024-03-24
2024-08-12
None - None

Keywords

  • CVSS.
  • Pentesters
  • MulVAL
  • Nmap
  • DQN

Abstract

In this research, we propose a revolutionary deep reinforcement learning-based methodology for automated penetration testing. The suggested method uses a deep Q-learning network to develop attack sequences that effectively exploit weaknesses in a target system. The method is tested in a virtual environment, and the findings indicate that it can identify vulnerabilities that manual penetration testing is unable to. A variety of tools, including Deep Q-learning network, MulVAL, Nmap, VirtualBox, Docker, National Vulnerability Database (NVD), and Common Vulnerability Scoring System (CVSS), are used in this work. The suggested method significantly outperforms current automated penetration testing methods. Our proposed methodology can detect flaws that manual penetration testing misses and can be modified (in terms of penalty values) to adapt to the updates of the target system (network) changes. Additionally, it has the potential to greatly enhance penetration testing's effectiveness and efficiency and could contribute to the increased security of computer systems. Experimental tests conducted in this work reveal the effectiveness of DQN automated penetration testing by utilizing the most effective attack vectors in the attack automation process

Recommended Citation

Jabr, I., Salman, Y., Shqair, M., & Hawash, A. (2025). Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach. An-Najah University Journal for Research - A (Natural Sciences), 39(1). https://doi.org/10.35552/anujr.a.39.1.2231
[1]I. Jabr, Y. Salman, M. Shqair, and A. Hawash, “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach,” An-Najah University Journal for Research - A (Natural Sciences), vol. 39, no. 1, Feb. 2025, doi: 10.35552/anujr.a.39.1.2231.
Jabr, Ismael, et al. “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach.” An-Najah University Journal for Research - A (Natural Sciences), vol. 39, no. 1, Feb. 2025. Crossref, https://doi.org/10.35552/anujr.a.39.1.2231.
1.Jabr I, Salman Y, Shqair M, Hawash A. Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach. An-Najah University Journal for Research - A (Natural Sciences) [Internet]. 2025 Feb;39(1). Available from: http://dx.doi.org/10.35552/anujr.a.39.1.2231
Jabr, Ismael, Yanal Salman, Motasem Shqair, and Amjad Hawash. “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach.” An-Najah University Journal for Research - A (Natural Sciences) 39, no. 1 (February 2025). https://doi.org/10.35552/anujr.a.39.1.2231.

Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach

معلومات المقال

2023-09-11
2024-03-24
2024-08-12
None - None

الكلمات الإفتتاحية

  • CVSS.
  • Pentesters
  • MulVAL
  • Nmap
  • DQN

الملخص

In this research, we propose a revolutionary deep reinforcement learning-based methodology for automated penetration testing. The suggested method uses a deep Q-learning network to develop attack sequences that effectively exploit weaknesses in a target system. The method is tested in a virtual environment, and the findings indicate that it can identify vulnerabilities that manual penetration testing is unable to. A variety of tools, including Deep Q-learning network, MulVAL, Nmap, VirtualBox, Docker, National Vulnerability Database (NVD), and Common Vulnerability Scoring System (CVSS), are used in this work. The suggested method significantly outperforms current automated penetration testing methods. Our proposed methodology can detect flaws that manual penetration testing misses and can be modified (in terms of penalty values) to adapt to the updates of the target system (network) changes. Additionally, it has the potential to greatly enhance penetration testing's effectiveness and efficiency and could contribute to the increased security of computer systems. Experimental tests conducted in this work reveal the effectiveness of DQN automated penetration testing by utilizing the most effective attack vectors in the attack automation process

Recommended Citation

Jabr, I., Salman, Y., Shqair, M., & Hawash, A. (2025). Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach. An-Najah University Journal for Research - A (Natural Sciences), 39(1). https://doi.org/10.35552/anujr.a.39.1.2231
[1]I. Jabr, Y. Salman, M. Shqair, and A. Hawash, “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach,” An-Najah University Journal for Research - A (Natural Sciences), vol. 39, no. 1, Feb. 2025, doi: 10.35552/anujr.a.39.1.2231.
Jabr, Ismael, et al. “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach.” An-Najah University Journal for Research - A (Natural Sciences), vol. 39, no. 1, Feb. 2025. Crossref, https://doi.org/10.35552/anujr.a.39.1.2231.
1.Jabr I, Salman Y, Shqair M, Hawash A. Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach. An-Najah University Journal for Research - A (Natural Sciences) [Internet]. 2025 Feb;39(1). Available from: http://dx.doi.org/10.35552/anujr.a.39.1.2231
Jabr, Ismael, Yanal Salman, Motasem Shqair, and Amjad Hawash. “Penetration Testing and Attack Automation Simulation: Deep Reinforcement Learning Approach.” An-Najah University Journal for Research - A (Natural Sciences) 39, no. 1 (February 2025). https://doi.org/10.35552/anujr.a.39.1.2231.

An-Najah National University
Nablus, Palestine
P.O. Box
7, 707
Fax
(970)(9)2345982
Tel.
(970)(9)2345560
(970)(9)2345113/5/6/7-Ext. 2628
E-mail
[email protected]
EIC
Prof. Waleed Sweileh

An-Najah University Journal for Research - A (Natural Sciences) by An-Najah University, Nablus, Palestine is licensed under CC BY-NC 4.0