A unitary design is a collection of unitary matrices that approximate the entire unitary group, much like a spherical design approximates the entire unit sphere. In this paper, we use irreducible representations of the unitary group to find a general lower bound on the size of a unitary t-design in U(d), for any d and t. We also introduce the notion of a unitary code-a subset of U(d) in which the trace inner product of any pair of matrices is restricted to only a small number of distinct absolute values-and give an upper bound for the size of a code with s inner product values in U(d), for any d and s. These bounds can be strengthened when the particular inner product values that occur in the code or design are known. Finally, we describe some constructions of designs: we give an upper bound on the size of the smallest weighted unitary t-design in U(d), and we catalogue some t-designs that arise from finite groups.