Pairwise Application Log Classification

This paper proposes a method for labeling big data applications based on their execution logs. This method allows big data cluster operators to identify and label applications without relying on the application names assigned by cluster users. Our method relies on pairwise classification and does not require extensive training data. We have evaluated the pairwise classification technique with 13 big data applications. Our method works with high accuracy for new application types that were not observed during the model training. Our classifier achieves 83% accuracy.