The Zel'dovich approximation, combined with an initial spectrum, appears to yield a surprisingly good prescription of the large-scale matter distribution for the evolution of structure in the Universe; in particular, it describes the evolution of structure fairly accurately well into the non-linear regime, and is thus superior to the standard Eulerian linear perturbation theory. We calculate the evolution of the power spectrum P(k, a) of the density field in the Zel'dovich approximation, which can be reduced to a single one-dimensional integral. The resulting expression reproduces the result from linear perturbation theory for small values of the cosmic scale factor a. On the other hand, the power spectrum as obtained from the Zel'dovich approximation predicts the generation of power on small scales, mainly as a result of the formation of compact structures and caustics. In fact, it is shown that, for k-->infinity, P(k, a) behaves like k(-3) on scales for which dissipative processes are negligible; this asymptotic behaviour is not an artefact of the Zel'dovich approximation, but is due to the occurrence of pancakes. We evaluate the power spectrum for standard hot dark matter (HDM) and cold dark matter (CDM) spectra; in the latter case, we employ the truncated Zel'dovich approximation which has been shown previously to yield much better agreement with the results from N-body simulations in cases where the primordial power spectrum contains large amounts of power on small scales. We obtain a simple fitting formula for the smoothing scale used in the truncated Zel'dovich approximation.