Towards Reliable Drift Detection and Explanation in Text Data

When delivered to the market, machine learning models face new data which are possibly subject to novel characteristics - a phenomenon known as concept drift. As this might lead to performance degradation, it is necessary to detect such drift and, if required, adapt the model accordingly. While a variety of drift detection and adaptation methods exists for standard vectorial data, a suitable treatment of text data is less researched. In this work we present a novel approach which detects and explains drift in text data based on their representation via transformer embeddings. In a nutshell, the method generates suitable statistical features from the original distribution and the possibly shifted variation. Based on these representations, drift scores can be assigned to individual data points, allowing a visualization and human-readable characterization of the type of drift. We demonstrate the approach's effectiveness in reliably detecting drift in several experiments.