Less is More: Revisiting the Gaussian Mechanism for Differential Privacy

Differential privacy (DP) via output perturbation has been a de facto standard for releasing query or computation results on sensitive data. Different variants of the classic Gaussian mechanism have been developed to reduce the magnitude of the noise and improve the utility of sanitized query results. However, we identify that all existing Gaussian mechanisms suffer from the curse of full-rank covariance matrices, and hence the expected accuracy losses of these mechanisms equal the trace of the covariance matrix of the noise. Particularly, for query results in R-M (or R-MxN in a matrix form), in order to achieve (epsilon, delta)-DP, the expected accuracy loss of the classic Gaussian mechanism, that of the analytic Gaussian mechanism, and that of the MatrixVariate Gaussian (MVG) mechanism are lower bounded by C-C(Delta(2) f)(2), C-A(Delta(2) f)(2), and C-M(Delta 2 f)(2), respectively, where C-C = 2ln(1.25/delta)epsilon(2) M, C-A = (Phi(-1)(delta))(2) +epsilon/epsilon(2) M, C-M = 5/4Hr+ 1/4 H-r,1/2/2 epsilon MN, Delta(2) f is the l(2) sensitivity of the query function f, Phi(center dot)(-1) is the quantile function of the standard normal distribution, H-r is the rth harmonic number, and H-r,H-1/2 is the rth harmonic number of order 1/2. To lift this curse, we design a Rank-1 SingularMultivariate Gaussian (R1SMG) mechanism. It achieves (epsilon, delta)-DP on query results in RM by perturbing the results with noise following a singular multivariate Gaussian distribution, whose covariance matrix is a randomly generated rank-1 positive semi-definite matrix. In contrast, the classic Gaussian mechanism and its variants all consider deterministic full-rank covariance matrices. Our idea is motivated by a clue from Dwork et al.'s seminal work on the classic Gaussian mechanism that has been ignored in the literature: when projecting multivariate Gaussian noise with a full-rank covariance matrix onto a set of orthonormal basis in R-M, only the coefficient of a single basis can contribute to the privacy guarantee. This paper makes the following technical contributions. (i) The R1SMG mechanisms achieves (epsilon, delta)-DP guarantee on query results in R-M, while its expected accuracy loss is lower bounded by C-R(Delta(2) f)(2), where C-R = 2/epsilon psi and psi = (delta Gamma( M-1/2)/root pi Gamma(M/2))(2/M-2). We show that CR is on a lower order of magnitude by at least M or MN compared with C-C, C-A, and C-M. Therefore, the expected accuracy loss of the R1SMG mechanism is on a much lower order compared with that of the classic Gaussian mechanism, of the analytic Gaussian mechanism, and of the MVG mechanism. (ii) Compared with other mechanisms, the R1SMG mechanism is more stable and less likely to generate noise with large magnitude that overwhelms the query results, because the kurtosis and skewness of the nondeterministic accuracy loss introduced by this mechanism is larger than that introduced by other mechanisms.