If G be a finite p-group and chi is a non-linear irreducible character of G, then chi(1) <= |G/Z(G)|(1/2). In [2], Fernandez-Alcober and Moreto obtained the relation between the character degree set of a finite p-group G and its normal subgroups depending on whether |G/Z(G)| is a square or not. In this paper we investigate the finite p-group G where for any normal subgroup N of G with G ' not less than or equal to N either N <= Z(G) or |NZ(G)/Z(G)| <= p and obtain some alternate characterizations of such groups. We find that if G is a finite p-group with |G/Z(G)| = p(2n+1) and G satisfies the condition that for any normal subgroup N of G either G ' not less than or equal to N or N <= Z(G), then cd(G) ={1, p(n)}. We also find that if G is a finite p-group with nilpotency class not equal to 3 and |G/Z(G)| = p(2n) and G satisfies the condition that for any normal subgroup N of G either G ' not less than or equal to N or |NZ(G)/Z(G)| <= p, then cd(G) subset of {1, p(n-1), p(n)}.