Given a field K of characteristic p > 0 and a natural number n, assuming that G is a permutation group acting on a set Omega with n elements, then KO is a permutation module for G in the natural way. If G is primitive and n <= 5 p, we will show that EndKG(K Omega) is always a symmetric Nakayama algebra unless p = 5 and n = 25. As a consequence, EndKG(K Omega) is always a symmetric Nakayama algebra if G is quasiprimitive, n < 4 p and 3 {' p - 1 when n = 3 p.