A rough path in a vector space V is a family (X',...,X-[p]), where X-1 is a classical path, and X-1, i greater than or equal to 2, are multi-integrals in a certain sense, p is related tv the roughness, so that the Ito map obtained by solving a differential equation driven by a rough path is a functional of the family (X-1,...,X-[p]). We study in this paper the calculus of variation for the Ito map by not only varying the classical path X-1, but also X-2. Almost all the sample paths of a continuous semimartingale are rough paths up to degree two in our sense, so that results in this paper can be applied to the Ito functionals on Wiener space.