Revisiting the Problem of Missing Values in High-Dimensional Data and Feature Selection Effect

Missing values are a prevalent challenge in mass spectrometry (MS) data, and many of the typical analysis approaches (e.g. regression approaches) follow a listwise deletion in the presence of missingness leading to information loss. To address missingness, numerous imputation methods (IMs) have been proposed. Nonetheless, the choice of method is of key importance both in relation to computational cost, especially for high dimensional data, as well as in relation to the impact on downstream data analyses. Despite the extensive published literature for utilizing distinct IMs tailored to specific missing value scenarios, there is scant research concerning the impact of IMs on feature selection. In this study, four computationally fast IMs (Zero, Mean, Median, and Expectation-Maximization) were considered on synthetically missing MS data for a range of scenarios, including different missing mechanisms (Missing at Random-MAR, Missing Completely at RandomMCAR, Missing Not at Random-MNAR) and percentages of missingness (10%, 20%, 50%). Least absolute shrinkage and selection operator (LASSO) regression was employed on the imputed data to examine how the choice of different IMs, under different scenarios of missingness, performed in terms of the choice of features and their estimates. We observed that all IMs considered, achieved high levels of accuracy performance at different scenarios of missingness. Also, LASSO regression results showed a certain level of agreement, as evidenced by the features that were commonly selected across different IMs for the same scenario of missingness. It must be noted that the magnitude of coefficients of the common selected features was influenced by the choice of IM. The findings from this simulation study provide valuable insights to analysts and researchers, highlighting that computationally efficient IMs can offer a good level of accuracy for missingness scenarios in high dimensional data. Acknowledging potential challenges, this study provides a foundation for further simulations to guide the choice of imputation approach for scenarios of high dimensional data in the presence of missingness.