Background Sedentary behavior and physical inactivity are primary factors contributing to college students' continued decline in health-related physical fitness (HRPF). This study aims to examine the associations between 24-h movement behavior and HRPF, and explore the dose-response relationship between the reallocation of movement behavior time and HRPF. Method In total, 1032 college students (649 males and 383 females) aged 19.5 years from China University of Mining and Technology were recruited to participate in the study in October 2023. The 24-h movement behavior was measured using reliable items from the 24-h Movement Behavior Questionnaire (reliability coefficients 0.68 to 0.97). HRPF was assessed using the Fitness Test Battery, which measures BMI, vital capacity, sit-and-reach, stand-long-jump, pull-up/crunches, 50-m sprint, and the 800/1000-m run. The isotemporal substitution model was applied to evaluate the health effects of different movement components. Results The 24-h movement behavior significantly influenced HRPF (p < 0.01, R-2 = 0.16). A positive correlation was found between HRPF and the proportion of moderate-to-vigorous physical activity (MVPA) (beta(MVPA) = 1.03, p < 0.01) and learning sedentary (LSED) (beta(LSED) = 2.41, p < 0.05), while a negative correlation existed with sleep (beta(SLP) = - 3.52, p < 0.01). Significant reallocations of 30 min from sleep to MVPA (+ 0.57, p < 0.05) and from screening sedentary (SSED) to MVPA (+ 0.39, p < 0.05) were observed. No significant associations were found between HRPF and reallocating from light-intensity physical activity to MVPA. Conclusion In addition to MVPA, it is important to consider the health risks associated with SSED and excessive sleep and to maintain a balanced time structure of 24-h movement behavior to promote health.