Testosterone plays an important role in regulating fertility, muscle mass and bone density. Low bone mineral density has been reported in men with testosterone deficiency syndrome (TDS); however, there is limited research available on the prevalence of TDS. In this study, we aimed to determine the prevalence of osteoporosis and osteopenia in middle-aged and older men (n = 4707, age: 40-79 years) with TDS. Their total serum testosterone levels were measured, and TDS was defined as a testosterone level of <= 3.5 ng/mL. To assess bone mineral density, we used dual-energy X-ray absorptiometry to measure the density at the lumbar spine (L1-L4) and femoral neck (T-score: osteopenia <=-1.0; osteoporosis <=-2.5). We performed logistic regression analysis to calculate adjusted odds ratios (AORs) after adjusting for age, body mass index, physical activity and VO2peak (Volume oxygen peak). The results showed a significant difference in physical activity between men with TDS and those without TDS (NTDS) in both middle-aged and older age groups (p < 0.05). Among middle-aged men, the AOR for osteopenia in the lumbar spine was 1.2-fold higher (Confience interval, CI (Confidence interval), 1.017-2.698), and in the femoral neck, it was 1.3-fold higher (CI, 1.012-3.013) for individuals with TDS compared to those without TDS. In older men, the AOR for osteopenia in the femoral neck increased by 1.4-fold (CI, 1.029-2.530). In conclusion, our findings suggest that middle-aged men with TDS have a higher likelihood of experiencing osteopenia in both the lumbar spine and femoral neck, while older men are more likely to have osteopenia, specifically in the femoral neck, but not osteoporosis.