Reduction of IDS false alarms using KNN classifier

Abstract

Intrusion detection is one of the important aspects in information security. Many commercial intrusion detection systems (IDSs) are available and are widely used by organizations. However, most of them suffer from the problem of high false alarm rate. Previous research reported that 90% of alarms from an IDS monitoring an operational network could be false alarms. This added heavy workload to security officers who are responsible for handling the alerts. If the security officers get tired of IDS alarms, they may become less sensitive to new alarms. In case a true attack occurs and get caught by the IDS, the officers may not be aware of it. Eventually the usability of IDSs is degraded. There is a great demand for a solution to this problem. In this thesis, we propose a new method to reduce the number of false alarms. We argue that when the network is experiencing intrusions, the IDS will trigger some alarms that possess different patterns from attack-free situation. On the other hand, if the incoming alarms align with the normal patterns, these alarms can be classified as false alarms. We model the alarm patterns of IDSs under an attack-free situation and detect anomalies from incoming alarm streams using k-nearest-neighbor (kNN) classifier. Experiment using DARPA dataset showed that the classifier can achieve high false alarm reduction rate up to 93%. We further tested the reduction mechanism with live data collected from an operating network with simulated intrusions and the experiment showed that our approach successfully reduced 94% of false alarms. We believe that the proposed false alarm method will be applicable to most of the existing IDSs as a plug-in feature, especially signature-based IDS. An architecture to apply the false alarm filtering mechanism to IDSs is suggested to illustrate the relationship between the reduction process and the IDSs and how the false alarms are filtered real-time.

Date of Award	16 Jul 2007
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Lam For KWOK (Supervisor)