Robust deep reinforcement learning for personalized HVAC system

The primary goal of heating, ventilation, and air conditioning (HVAC) systems is to maximize energy efficiency while maintaining temperature in the desired comfortable range. However, most existing HVAC systems do not consider individual comfort, resulting in energy waste and thermal discomfort. In this context, we propose a deep reinforcement learning (DRL)-based HVAC control framework with two objectives: personalization and increased robustness. For personalization, we proposed a data-driven personalized overall thermal sensation (OTS) prediction model that utilizes individual data, including OTS and environmental variables. Human experiments were conducted with four participants to collect individual data, and a machine learning-based personalized OTS prediction model was developed. For robustness, the soft actor-critic (SAC) algorithm was used for HVAC control, demonstrating good performance in terms of exploration and robustness to uncertainties. The proposed method's effectiveness in terms of personalization and robustness was validated through simulations. Our personalized OTS model improves the thermal comfort experienced by individuals by an average of 0.43 points over the predicted mean value (PMV). Furthermore, the SAC algorithm achieves more robust performance in terms of experienced thermal comfort than other algorithms by an average of 24% under uncertainties.