Skip to main content
editor.iijri@gmail.com
8007937666
e-ISSN: applied
The Intellecta International Journal of Research and Innovation

The Intellecta International Journal of Research and Innovation

LoginTrack Your Paper
Published

A PREVALENCE ADJUSTED AND PROTOCOL-SPECIFIC EVALUATION OF MACHINE LEARNING-BASED INTRUSION DETECTION SYSTEMS

Published in Oct-Dec 2026 (Vol. 1, Issue 1, 2026)

A PREVALENCE ADJUSTED AND PROTOCOL-SPECIFIC EVALUATION OF MACHINE LEARNING-BASED INTRUSION DETECTION SYSTEMS - Issue cover

Keywords

Intrusion detectionevaluation methodologyprotocol stratificationbase rateprecisionclass priorreporting standards.

Abstract

Machine learning intrusion detection is evaluated almost entirely through metrics computed across an entire test partition. This paper argues that the practice has two defects and demonstrates both with measurements. The first defect is protocol-level concealment: network protocols carry attack traf ic in very dif erent proportions, so an aggregate metric averages over problems of unequal base rate, and a detector that fails on a minority protocol can still report a respectable aggregate figure. The second defect is prevalence distortion: precision depends on the base rate of the positive class, and benchmark corpora carry attack prevalences one to two orders of magnitude above those of operational networks, so reported precision overstates what a deployed system will achieve. Six experiments were conducted, crossing three detectors with two benchmark corpora and evaluating 72,856 held-out records. A Random Forest reporting an aggregate recall of 0.7626 was found to attain a recall of 0.0361 within the ICMP subset, a dif erence of a factor of 21.1 that no aggregate statistic exposes. Detection quality across the five assessable protocol subsets correlated with subset attack prevalence at r = 0.7773. Holding measured sensitivity and specificity fixed, a precision of 0.9793 at benchmark prevalence corresponds to 0.3415 at an operational prevalence of one percent. The paper closes with a reporting protocol that addresses both defects and costs no additional data, training, or computation.

References

  1. [1]S. Axelsson, “The base rate fallacy and the difficulty of intrusion detection,” ACM Transactions on Information and System Security, vol. 3, no. 3, pp. 186–205, 2000.
  2. [2]T. A. Tang, L. Mhamdi, D. McLernon, S. A. R. Zaidi, and M. Ghogho, “Deep learning approach for network intrusion detection in software defined networking,” in Proceedings of the International Conference on Wireless Networks and Mobile Communications (WINCOM), 2016, pp. 258–263.
  3. [3]C. Yin, Y. Zhu, J. Fei, and X. He, “A deep learning approach for intrusion detection using recurrent neural networks,” IEEE Access, vol. 5, pp. 21954–21961, 2017.
  4. [4]R. C. Aygun and A. G. Yavuz, “Network anomaly detection with stochastically improved autoencoder based models,” in Proceedings of the IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud), 2017, pp. 193–198.
  5. [5]S. Naseer et al., “Enhanced network anomaly detection based on deep neural networks,” IEEE Access, vol. 6, pp. 48231–48246, 2018.
  6. [6]F. A. Khan and A. Gumaei, “A comparative study of machine learning classifiers for network intrusion detection,” in Proceedings of the 5th International Conference on Artificial Intelligence and Security (ICAIS), 2019, pp. 75–86.
  7. [7]R. K. Malaiya, D. Kwon, S. C. Suh, H. Kim, I. Kim, and J. Kim, “An empirical evaluation of deep learning for network anomaly detection,” IEEE Access, vol. 7, pp. 140806–140817, 2019.
  8. [8]X. Zhou, Y. Hu, W. Liang, J. Ma, and Q. Jin, “Variational LSTM enhanced anomaly detection for industrial big data,” IEEE Transactions on Industrial Informatics, vol. 17, pp. 3469–3477, 2020.
  9. [9]B. Min, J. Yoo, S. Kim, D. Shin, and D. Shin, “Network anomaly detection using memory augmented deep autoencoder,” IEEE Access, vol. 9, pp. 104695–104706, 2021.
  10. [10]J. Jeong, S. Park, J. Lim, J. Kang, D. Shin, and D. Shin, “A study on network anomaly detection using fast persistent contrastive divergence,” Symmetry, vol. 16, no. 9, Art. no. 1220, 2024.
  11. [11]R. Sommer and V. Paxson, “Outside the closed world: On using machine learning for network intrusion detection,” in Proceedings of the IEEE Symposium on Security and Privacy, 2010, pp. 305–316.
  12. [12]M. Ring, S. Wunderlich, D. Scheuring, D. Landes, and A. Hotho, “A survey of network based intrusion detection data sets,” Computers & Security, vol. 86, pp. 147–167, 2019.
  13. [13]M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” in Proceedings of the IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), 2009, pp. 1–6.
  14. [14]N. Moustafa and J. Slay, “UNSW NB15: A comprehensive data set for network intrusion detection systems,” in Proceedings of the Military Communications and Information Systems Conference (MilCIS), 2015, pp. 1–6.
  15. [15]L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.
  16. [16]F. Pedregosa et al., “Scikit learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
  17. [17]S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma, “Linformer: Self attention with linear complexity,” arXiv preprint arXiv:2006.04768, 2020.
  18. [18]A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 5998–6008.
  19. [19]A. Paszke et al., “PyTorch: An imperative style, high performance deep learning library,” in Advances in Neural Information Processing Systems, vol. 32, 2019, pp. 8024–8035.
  20. [20]D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proceedings of the International Conference on Learning Representations (ICLR), 2015.

Authors (1)

Varsha

Department of Computer Science...

View all publications →

Download Article

PDF

Best for printing and citation

File size: 0.8 MB
Format: PDF
ViewDownload

Article Information

Published in:

Oct-Dec 2026 (Vol. 1, Issue 1, 2026)

IIJRI110003

IIJRI-01-000007

24-34

2026-10-06

JATS XML

Article Impact

Views:31
Downloads:26

How to Cite

Varsha (2026). A PREVALENCE ADJUSTED AND PROTOCOL-SPECIFIC EVALUATION OF MACHINE LEARNING-BASED INTRUSION DETECTION SYSTEMS. The Intellecta International Journal of Research and Innovation, 1(1), 24-34. https://iijri.a1advancepublications.com/articles/IIJRI110003

Article Actions

Whatsapp