We consider interacting many-particle systems with quenched disorder having strong Griffiths singularities, which are characterized by the dynamical exponent, z, such as random quantum systems and exclusion processes. In several d=1 and d=2 dimensional problems we have calculated the inverse time scales, tau(-1), in finite samples of linear size, L, either exactly or numerically. In all cases, having a discrete symmetry, the distribution function, P(tau(-1),L), is found to depend on the variable, u=tau L--1(z), and to be universal given by the limit distribution of extremes of independent and identically distributed random numbers. This finding is explained in the framework of a strong disorder renormalization group approach when, after fast degrees of freedom are decimated out, the system is transformed into a set of noninteracting localized excitations. The Frechet distribution of P(tau(-1),L) is expected to hold for all random systems having a strong disorder fixed point, in which the Griffiths singularities are dominated by disorder fluctuations.