Small models, big impact: A review on the power of lightweight Federated Learning

Federated Learning (FL) enhances Artificial Intelligence (AI) applications by enabling individual devices to collaboratively learn shared models without uploading local data with third parties, thereby preserving privacy. However, implementing FL in real-world scenarios presents numerous challenges, especially with IoT devices with limited memory, diverse communication conditions, and varying computational capabilities. The research community is turning to lightweight FL, the new solutions that optimize FL training, inference, and deployment to work efficiently on IoT devices. This paper reviews lightweight FL, systematically organizing and summarizing the related techniques based on its workflow. Finally, we indicate potential problems in this area and suggest future directions to provide valuable insights into the field.