STM Journals Peer-reviewed research from Consortium e-Learning Network

Journal of Artificial Intelligence Research & Advances Review Article

Adversarial Attacks on Machine Learning Models in Cybersecurity: A Systematic Literature Review

  1. Sanjay Singh Department of Computer Science, Shaheed Sukhdev College of Business Studies, University of Delhi, New Delhi
  2. Anamika Gupta Department of Computer Science, Shaheed Sukhdev College of Business Studies, University of Delhi, New Delhi

Abstract

Adversarial machine learning (AML) is a field that is growing swiftly, especially as machine learning models are employed more and more in places where security is critical. This review goes into great depth over 746 publications from the Scopus database, with an emphasis on the connection between AML and network security. Using Biblioshiny and Scopus tools, we looked at trends in publications, study fields, productive authors, collaboration networks, and theme concentrations. Thirteen charts show how AML research has developed over time by highlighting notable contributions, main journals, popular keywords, and citation trends. Our research demonstrates that the topic is multidisciplinary and has research centers all around the world. It also shows that scholars are not working together as much anymore and that the focus is moving from pure attack modeling to robustness and interpretability. At the end of the study, some major gaps are pointed out, such as the clichés about real correspondent implementation and ethical governance. The report also offers approaches for future research to make the AML research ecosystem stronger and more accessible to everyone.

Keywords

References (51)

  1. Babatunde LA, Etim ED, Essien IA, Cadet E, Ajayi JO, Erigha ED, et al. Adversarial Machine Learning in Cybersecurity: Vulnerabilities and Defense Strategies. Journal of Frontiers in Multidisciplinary Research. 2020;1(2):31-45. doi:10.54660/.jfmr.2020.1.2.31-45
  2. Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Xiao C, et al. Robust Physical-World Attacks on Deep Learning Visual Classification. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:1625-1634. doi:10.1109/cvpr.2018.00175
  3. Biggio B, Roli F. Wild Patterns. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2018:2154-2156. doi:10.1145/3243734.3264418
  4. Akhtar N, Mian A. Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey. IEEE Access. 2018;6:14410-14430. doi:10.1109/access.2018.2807385
  5. Zhang J, Hu J, Liu J. Neural network with multiple connection weights. Pattern Recognition. 2020;107:107481. doi:10.1016/j.patcog.2020.107481
  6. Chen X, Liu C, Li B, Lu K, Song D. Targeted backdoor attacks on deep learning systems using data poisoning. [Preprint]. 2017. arXiv:1712.05526. doi:10.48550/arXiv.1712.05526.
  7. Dilhara M, Ketkar A, Dig D. Understanding Software-2.0. ACM Transactions on Software Engineering and Methodology. 2021;30(4):1-42. doi:10.1145/3453478
  8. Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. [Preprint]. 2016. arXiv:1610.02136. doi:10.48550/arXiv.1610.02136.
  9. Dong Y, Liao F, Pang T, Su H, Zhu J, Hu X, et al. Boosting Adversarial Attacks with Momentum. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:9185-9193. doi:10.1109/cvpr.2018.00957
  10. He J, Li X, Liao L, Song D, Cheung W. Inferring a Personalized Next Point-of-Interest Recommendation Model with Latent Behavior Patterns. Proceedings of the AAAI Conference on Artificial Intelligence. 2016;30(1). doi:10.1609/aaai.v30i1.9994
  11. Gu T, Dolan-Gavitt B, Garg S. Badnets: Identifying vulnerabilities in the machine learning model supply chain. [Preprint]. 2017. arXiv:1708.06733. doi:10.48550/arXiv.1708.06733.
  12. Liu Y, Chen X, Liu C, Song D. Delving into transferable adversarial examples and black-box attacks. [Preprint]. 2016. arXiv:1611.02770. doi:10.48550/arXiv.1611.02770.
  13. Carlini N, Wagner D. Towards Evaluating the Robustness of Neural Networks. 2017 IEEE Symposium on Security and Privacy (SP). 2017:39-57. doi:10.1109/sp.2017.49
  14. Demontis A, Melis M, Pintor M, Jagielski M, Biggio B, Oprea A, et al. Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In: Proceedings of the 28th USENIX Security Symposium (USENIX Security 19). Berkeley (CA): USENIX Association; 2019. p. 321–338.
  15. Wang X, Li J, Kuang X, Tan YA, Li J. The security of machine learning in an adversarial setting: A survey. Journal of Parallel and Distributed Computing. 2019;130:12-23. doi:10.1016/j.jpdc.2019.03.003
  16. Ilyas A, Santurkar S, Tsipras D, Engstrom L, Tran B, Madry A. Adversarial examples are not bugs, they are features. In: Wallach HM, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox E, Garnett R, editors. Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS 2019); 2019 Dec 8–14; Vancouver, BC, Canada. Red Hook (NY): Curran Associates Inc.; 2019. P. 125–136.
  17. Jagielski M, Oprea A, Biggio B, Liu C, Nita-Rotaru C, Li B. Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning. 2018 IEEE Symposium on Security and Privacy (SP). 2018:19-35. doi:10.1109/sp.2018.00057
  18. Li Y, Huang H, Guo X, Yuan Y. An Empirical Study on Group Fairness Metrics of Judicial Data. IEEE Access. 2021;9:149043-149049. doi:10.1109/access.2021.3122443
  19. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A. Towards deep learning models resistant to adversarial attacks. [Preprint]. 2017. arXiv:1706.06083. doi:10.48550/arXiv.1706.06083.
  20. Nasr M, Shokri R, Houmansadr A. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. 2019 IEEE Symposium on Security and Privacy (SP). 2019:739-753. doi:10.1109/sp.2019.00065
  21. Kurita K, Vyas N, Pareek A, Black AW, Tsvetkov Y. Measuring Bias in Contextualized Word Representations. Proceedings of the First Workshop on Gender Bias in Natural Language Processing. 2019:166-172. doi:10.18653/v1/w19-3823
  22. Su J, Vargas DV, Sakurai K. One Pixel Attack for Fooling Deep Neural Networks. IEEE Transactions on Evolutionary Computation. 2019;23(5):828-841. doi:10.1109/tevc.2019.2890858
  23. Li J, Gao J, Jiang Q, He G. Adversarial Defense Networks via Gaussian Noise and RBF. Lecture Notes in Computer Science. 2021:494-504. doi:10.1007/978-3-030-78609-0_42
  24. Xiao C, Li B, Zhu JY, He W, Liu M, Song D. Generating Adversarial Examples with Adversarial Networks. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. 2018:3905-3911. doi:10.24963/ijcai.2018/543
  25. Sadeghi AR, Wachsmann C, Waidner M. Security and privacy challenges in industrial internet of things. Proceedings of the 52nd Annual Design Automation Conference. 2015:1-6. doi:10.1145/2744769.2747942
  26. Cheng G, Sun X, Li K, Guo L, Han J. Perturbation-Seeking Generative Adversarial Networks: A Defense Framework for Remote Sensing Image Scene Classification. IEEE Transactions on Geoscience and Remote Sensing. 2022;60:1-11. doi:10.1109/tgrs.2021.3081421
  27. Alfakih T, Hassan MM, Gumaei A, Savaglio C, Fortino G. Task Offloading and Resource Allocation for Mobile Edge Computing by Deep Reinforcement Learning Based on SARSA. IEEE Access. 2020;8:54074-54084. doi:10.1109/access.2020.2981434
  28. Zhao ZQ, Zheng P, Xu ST, Wu X. Object Detection With Deep Learning: A Review. IEEE Transactions on Neural Networks and Learning Systems. 2019;30(11):3212-3232. doi:10.1109/tnnls.2018.2876865
  29. Li L. Comprehensive survey on adversarial examples in cybersecurity: Impacts, challenges, and mitigation strategies. [Preprint]. 2024. arXiv:2412.12217. doi:10.48550/arXiv.2412.12217.
  30. Papernot N, McDaniel P, Jha S, Fredrikson M, Celik ZB, Swami A. The Limitations of Deep Learning in Adversarial Settings. 2016 IEEE European Symposium on Security and Privacy (EuroS&P). 2016:372-387. doi:10.1109/eurosp.2016.36
  31. Jia R, Liang P. Adversarial Examples for Evaluating Reading Comprehension Systems. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017:2021-2031. doi:10.18653/v1/d17-1215
  32. Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. [Preprint]. 2014. arXiv:1412.6572. doi:10.48550/arXiv.1412.6572.
  33. Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. Adversarial attacks on medical machine learning. Science. 2019;363(6433):1287-1289. doi:10.1126/science.aaw4399
  34. Pei K, Cao Y, Yang J, Jana S. DeepXplore. Proceedings of the 26th Symposium on Operating Systems Principles. 2017:1-18. doi:10.1145/3132747.3132785
  35. Cortes-Perez N, Torres-Mendez LA. A Low-Cost Mirror-Based Active Perception System for Effective Collision Free Underwater Robotic Navigation. 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2016:61-68. doi:10.1109/cvprw.2016.15
  36. Xie S, Girshick R, Dollar P, Tu Z, He K. Aggregated Residual Transformations for Deep Neural Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:5987-5995. doi:10.1109/cvpr.2017.634
  37. Nelson B, Barreno M, Chi FJ, Joseph AD, Rubinstein BIP, Saini U, et al. Exploiting machine learning to subvert your spam filter. In: Monrose F, editor. Proceedings of the First USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET ‘08); 2008 Apr 15; San Francisco, CA, USA. Berkeley (CA): USENIX Association; 2008.
  38. Papernot N, McDaniel P, Sinha A, Wellman MP. SoK: Security and Privacy in Machine Learning. 2018 IEEE European Symposium on Security and Privacy (EuroS&P). 2018:399-414. doi:10.1109/eurosp.2018.00035
  39. Sharif M, Bhagavatula S, Bauer L, Reiter MK. Accessorize to a Crime. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016:1528-1540. doi:10.1145/2976749.2978392
  40. Shokri R, Shmatikov V. Privacy-Preserving Deep Learning. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015:1310-1321. doi:10.1145/2810103.2813687
  41. Song C, Ristenpart T, Shmatikov V. Machine Learning Models that Remember Too Much. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017:587-601. doi:10.1145/3133956.3134077
  42. Gan S, Zhang C, Qin X, Tu X, Li K, Pei Z, et al. CollAFL: Path Sensitive Fuzzing. 2018 IEEE Symposium on Security and Privacy (SP). 2018:679-696. doi:10.1109/sp.2018.00040
  43. Tramèr F, Carlini N, Brendel W, Madry A. On adaptive attacks to adversarial example defenses. Adv Neural Inf Process Syst. 2020;33:1633–1645.
  44. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, et al. Intriguing properties of neural networks. [Preprint]. 2013. arXiv:1312.6199. doi:10.48550/arXiv.1312.6199.
  45. Recht B, Roelofs R, Schmidt L, Shankar V. Do ImageNet classifiers generalize to ImageNet? In: Chaudhuri K, Salakhutdinov R, editors. Proceedings of the 36th International Conference on Machine Learning (ICML); 2019 Jun 9-15; Long Beach, CA, USA. Proc Mach Learn Res. 2019;97:5389-5400.
  46. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning requires rethinking generalization. [Preprint]. 2016. arXiv:1611.03530. doi:10.48550/arXiv.1611.03530.
  47. Zhang H, Chen H, Song Z, Boning D, Dhillon IS, Hsieh CJ. The limitations of adversarial training and the blind-spot attack. [Preprint]. 2019. arXiv:1901.04684. doi:10.48550/arXiv.1901.04684.
  48. Xu H, Caramanis C, Mannor S. Robustness and regularization of support vector machines. J Mach Learn Res. 2009;10:1485-1510.
  49. Ren K, Zheng T, Qin Z, Liu X. Adversarial Attacks and Defenses in Deep Learning. Engineering. 2020;6(3):346-360. doi:10.1016/j.eng.2019.12.012
  50. Ruiz N, Bargal SA, Sclaroff S. Disrupting deepfakes: adversarial attacks against conditional image translation networks and facial manipulation systems. [Preprint]. 2020 Mar 3. arXiv:2003.01279. doi:10.48550/arXiv.2003.01279.
  51. Aria M, Cuccurullo C. bibliometrix : An R-tool for comprehensive science mapping analysis. Journal of Informetrics. 2017;11(4):959-975. doi:10.1016/j.joi.2017.08.007
Support