Towards healthy and cost-effective indoor environment management in smart homes: A deep reinforcement learning approach

Indoor environmental quality is an important issue since people spend most of their time indoors. This paper aims to develop an autonomous indoor environment management approach to ensure a healthy indoor environment with minimized energy cost via the optimal control of ventilation system and heating/cooling system in smart homes. Due to the uncertainties of weather conditions, electricity price and home occupancy, as well as the complex interaction between indoor air quality and indoor thermal environment, it is challenging to develop an efficient control strategy. To address this challenge, the indoor environment management problem is formulated as a Markov decision process, and then a deep reinforcement learning control strategy, which combines double deep Q network with prioritized experience replay mechanism, is proposed to solve the Markov decision process. The proposed approach can make adaptive control decisions based on the current observations without requiring any forecast information of system uncertainties. Control performance under different scenarios show the proposed approach has good adaptability to the variation of weather conditions, electricity prices, home occupancy patterns and indoor temperature requirements. Moreover, the proposed approach is compared with a double deep Q network-based approach and a model predictive control-based approach. Comparison results show that the proposed approach reduces the average daily energy cost by 3.51% and 8.56% in winter scenarios and 4.05%, 7.88% in summer scenarios while achieving the smallest indoor air quality and temperature violations.