The concept of the smart city, which has recently gained prominence as a new urbanization model, represents a practical approach to achieving a balance between economic development and environmental protection. This article analyzes the effect of upgrading urban industrial structures on green total factor productivity (GTFP) and addresses a gap in evaluating the mediating role of smart city development. Utilizing panel data from Chinese prefecture-level cities, the study finds that internal advancements and diversification in industrial structures significantly enhance GTFP, while industrial structure rationalization displays a complex, non-linear correlation with GTFP. Moreover, smart city development is found to positively moderate the relationship between industrial structure and productivity, particularly in terms of outer-advancement, diversification, and rationalization. The moderating effect is evident in green technical efficiency rather than green technological innovation. Cluster analysis reveals significant effects in regions characterized by irrational, diversified, and naturally rational industrial structures. Additionally, the study identifies five impact mechanisms facilitated by smart cities, namely ICT, finance development, digitalization, smart governance, and social trust. These conclusions are of material significance to promote high-quality urban development.