Mobile communications have been undergoing a generational evolution at least every ten years, currently in the fifth-generation (5G) and 5G-Advanced decade. As 5G and 5G-Advanced become a commercial reality, there is already significant interest in systems beyond 5G i.e., the sixth generation (6G) of wireless systems, anticipated to officially launch in 2030. The applications of 5G, 5G-Advanced, and 6G require more spectrum featuring a diverse range of radio frequency bands, ranging from below 6 GHz up to 1 THz. This includes low-band spectrum for coverage (macro cells) offering low data rates, mid-band for both coverage and capacity (small cells) offering high data rates, and high-band and beyond spectrum for capacity only, offering ultra-high data rates. By utilizing the capacity layer (small cells), a mobile user will take a shorter time to exit the cell. However, this exposes the user to very high handover requests, ping-pong effects, and handover failures, which negatively impact the quality of service (QoS) and quality of experience (QoE). Additionally, the integration of Terrestrial Networks (TNs) and Non-Terrestrial Networks (NTNs) envisioned for 5G, 5G-Advanced and 6G networks has made mobility (handover) management far more challenging due to mobile end-devices and infrastructure dynamics. To address the handover challenge, we proposed an infrastructure-agnostic handover framework for auto-tuning self-optimization in the handover measurement stage and handover decision-making stage. This framework aims to optimize handover performance and improve QoS and QoE for mobile users in 5G, 5G-Advanced, and 6G wireless networks.