Heavy metal pollution has become an increasingly prominent problem. Consequently, adsorbents with high heavy metal removal efficiency using a convenient synthesis method have to be developed urgently. The magnesium carbonate hydroxide microsheets modified activated carbon fiber (MCH@ACF) was prepared by a facile in situ hydrothermal method. The results of the characteristic of MCH@ACF indicated that a large amount of magnesium carbonate hydroxide porous microsheets is densely arranged on the surface of ACF. The Mn2+, Cu2+, Pb2+, and Cr2O72- ions adsorption kinetics of the MCH@ACF could be fitted with the pseudo-second-order model, and the results indicate the fast adsorption rate. The Mn2+, Cu2+, and Pb2+ ions adsorption on the MCH@ACF follow the Langmuir model, while the Cr2O72- adsorption on the MCH@ACF follows the Freundlich model. The theoretical adsorption capacity of the Mn2+, Cu2+, Pb2+, and Cr2O72- ions on the MCH@ACF is 558.6 mg/g, 1591.7 mg/g, 1802.8 mg/g, and 125.7 mg/g, respectively. The heavy metal ions removal performance of MCH@ACF under different pH conditions has been studied. Furthermore, the adsorption mechanism could be ascribed to the binding of heavy metal ions with carbonate or hydroxyl group. The results indicated that the MCH@ACF can be used as the potential suitable candidate for heavy metal removal.