Intrusion detection system

In general, studies of intrusion detection report only one accuracy or F1 score calculated on the entire test set. Since TCP, UDP and ICMP toxicate attack traf ic in very dif erent ratios, an average such a figure overprofiles problems of varying dif iculty and may mask a detector that collapses on a complete protocol. We introduce a protocol-aware anomaly detector, based on the Linformer, a variation of transformer that scales its attention cost linearly with the input length, and we rank it on each protocol on two benchmark corpora. On NSL-KDD (148,517 records) and UNSW-NB15 (215,757 records after protocol filtering), six experiments are done on a fully crossed design comparing the Linformer with a Decision Tree and a random forest that has 72,856 held-out records. On NSL-KDD the Linformer reached an accuracy of 0.9823, an F1-score of 0.9816 and an AUC-ROC of 0.9987; on UNSW-NB15 the corresponding values were 0.9231, 0.9344 and 0.9832. Breaking down the results according to protocol revealed a bug that the cumulative figures disguised: only 3.61 percent of the ICMP attacks were identified by the Random Forest, which has a cumulative accuracy of 0.8784 on NSL-KDD, compared to 99.30 percent by the Linformer. The protocol subsets (among five that are assessable) F1 increased with attack prevalence (Pearson = 0.7773), indicating that a direct solution is per-protocol threshold calibration. Extrapolating the measured error rates onto the prevalence values determined that accuracy reduces to 0.3415 at 1% attack traf ic, indicating the detector should not be used as a blocker, despite the fact that at 0.9793, the prevalence matches the benchmark prevalence criterion.