In this paper, we construct center stable manifolds around unstable line solitary waves of the Zakharov-Kuznetsov equation on two dimensional cylindrical spaces RxT(L) (T-L = R/2 pi LZ). In the paper [39], center stable manifolds around unstable line solitary waves have been constructed excluding the case of critical speeds c is an element of {4n(2)/5L(2); n is an element of Z, n > 1}. In the case of critical speeds c, any neighborhood of the line solitary wave with speed c in the energy space includes solitary waves which are depend on the direction TL. To construct center stable manifolds including their solitary waves and to recover the degeneracy of the linearized operator around line solitary waves with critical speed, we prove the stability condition of the center stable manifold for critical speed by applying to the estimate of the 4th order term of a Lyapunov function in [37] and [38].