Outlier Detection by means of Monte Carlo Estimation including resistant Scale Estimation

The identification of outliers in measurement data is hindered if they are present in leverage points as well as in rest of the data. A promising method for their identification is the Monte Carlo estimation (MCE), which is subject of the present investigation. In MCE the data are searched for data subsamples without leverage outliers and with few (or no) non-leverage outliers by a random generation of subsamples. The required number of subsamples by which several of such subsamples are generated with a given probability is derived. Each generated subsample is rated based on the residuals resulting from an adjustment. By means of a simulation it is shown that a least squares adjustment is suitable. For the rating of the subsamples, the sum of squared residuals is used as a measure of the fit. It is argued that this (unweighted) sum is also appropriate if data have unequal weights. An investigation of the robustness of a final Bayes estimation with the result of the Monte Carlo search as prior information reveals its inappropriateness. Furthermore, the case of an unknown variance factor is considered. A simulation for different scale estimators for the variance factor shows their impracticalness. A new resistant scale estimator is introduced which is based on a generalisation of the median absolut deviation. Taking into account the results of the investigations, a new procedure for MCE considering a scale estimation is proposed. Finally, this method is tested by simulation. MCE turns out to be more reliable in the identification of outliers than a conventional resistant estimation method.