Training-free approach to constructing ensemble of local experts

Conventional machine learning methods assume that a central server holds the entire training dataset for training a global model. However, in many real-world situations, datasets are collected from individual local nodes in distributed environments. Gathering these datasets often poses challenges owing to various practical constraints such as data security and privacy issues, storage limits, and transmission costs. When data distributions differ among local nodes, local experts trained on individual local nodes exhibit poor generalization performance for new instances. In this study, we propose a training-free method for constructing an ensemble of local experts without accessing datasets. For a query instance, we apply uncertainty quantification and out- of-distribution (OOD) detection to calculate the uncertainty and OOD scores of the predictions from individual models. We then determine the weights of the models by assigning a lower weight to those with higher uncertainty and OOD scores. Using these weights, the predictions of the individual models are aggregated to obtain a final prediction. In contrast to existing methods that impose specific application requirements, the proposed method can be applied in situations in which only individual predictions from local models are available for use when making a prediction for a query instance. We evaluate the effectiveness of the proposed method using image classification benchmark datasets.