To address the low energy density and potential safety issues of modern lithium-ion batteries (LIBs), all-solid-state lithium batteries (ASSLBs) with solid-state electrolytes (SSEs) have emerged as a highly promising option. Among different SSEs, inorganic electrolytes (IEs) are the most probable to replace organic liquid electrolytes because of their relatively high lithium ionic conductivity and wide cell voltage window. Nevertheless, IEs encounter challenges such as elevated interfacial resistance, limited ionic conductivity at room temperature, and air instabilities. Different hybrid solid electrolytes, including organic-inorganic hybrid solid electrolytes (OIHSEs) and inorganic composite electrolytes (ICEs), have been developed to overcome these difficulties. While OIHSEs have been reviewed thoroughly, ICEs have been reviewed rarely, despite their crucial role in the advancement of ASSLBs. This review focuses on the synthesis methodologies, structures, compositions, and electrochemical performance of ICEs, providing a comprehensive overview of the present state-of-art ICEs, with constructive conclusions and perspectives for different ICEs on varied purposes. Ultimately, this review aims to shed light on potential research directions for the designing and building of practical ASSLBs with varying ICEs, thereby promoting the energy storage application of ASSLBs.