We study differentiability properties of the Riesz potential,with kernel of homogeneity 2 - d in R-d, for d >= 3, of a finite Borel measure. In the plane we consider the logarithmic potential of a finite Borel measure. We introduce a notion of differentiability in the capacity sense, where capacity is the Newtonian capacity in dimension d >= 3 and the Wiener capacity in the plane. We require that the first order remainder at a point is small when measured by means of a normalized weak capacity "norm" in balls of small radii centered at the point. This implies L-P differentiability in the Calderon-Zygmund sense for 1 <= p < d/d-2. If d >= 3, we show that the Riesz potential of a finite Borel measure is differentiable in the capacity sense except for a set of zero C-1-harmonic capacity. The result is sharp and depends on deep results in non-doubling Calderon-Zygmund theory. In the plane the situation is different. Surprisingly there are two distinct notions of differentiability in the capacity sense. For each of them we obtain the best possible result on the size of the exceptional set in terms of Hausdorff measures. We obtain, for d >= 3, results on Peano second order differentiability in the sense of capacity with exceptional sets of zero Lebesgue measure. Finally, as an application, we find a new proof of the well-known fact that the equilibrium measure is singular with respect to the Lebesgue measure.
