It is known that perturbative invariants of rational homology 3-spheres can be constructed by using arithmetic perturbative expansion of quantum invariants of them. However, we could not make arithmetic perturbative expansion of quantum invariants for 3-manifolds with positive Betti numbers by the same method. In this paper, we explain how to make arithmetic perturbative expansion of quantum SO(3) invariants of 3-manifolds with the first Betti number 1. Further, motivated by this expansion, we construct perturbative invariants of such 3-manifolds. We show some properties of the perturbative invariants, which imply that their coefficients are independent invariants.