Parallelizing Count-Min Sketch Algorithm on Multi-core Processors

In this paper, we present a novel method that exploits the great parallel capability of multi-cores to speed up the famous Count-Min sketch algorithm. The proposed parallel Count-Min sketch algorithm equally distributes the input data stream into sub-threads which use the original Count-Min sketch algorithm to process the sub-streams. The counters in each local Count-Min sketch with frequency increments exceeding a pre-defined threshold are sent to a merging thread which is able to return the estimated frequencies satisfying the (epsilon, delta)-approximation requirement. Experiments with real traffic traces demonstrate the excellent performance as well as the effects of parameters. The parallel Count-Min sketch algorithm achieves near-linear speedup at the cost of greater memory use.