Community-based anomaly detection

Network behaviour anomaly detection systems can detect zero-day attacks and work even with encrypted traffic. They maintain a model of normal behaviour and report any deviation as anomaly. Typically, a separated model for each host is generated or there is one model for the whole network. The model of normal can be built for the whole network or for each network host separately. The per host models suffer from a small amount of noisy data as the behaviour of a single user is typically not very stable. The single model for the whole network is more robust to fluctuations, but it is trying to find a normal behaviour of a group of hosts with possibly diverse behaviour. We propose a method for clustering network hosts based on their network behaviour to create groups of hosts that behave similarly. The anomaly detection models trained on such groups of network hosts are more robust to fluctuations of the behaviour of individual hosts when compared to the per host models. It is able to detect finer anomalies (e.g. stealthy data ex-filtration) that would be otherwise hidden by modelling diversely behaving network hosts together.