When measuring the form deviation of the gear tooth flank with laser interferometry, due to the limitation of the depth of field of the lens, focused and defocused regions exist in the interferogram simultaneously. The introduced errors that were generated by these defocused regions decrease the measurement accuracy. Therefore, the evaluation of definition for interferograms is necessary fur image processing. However, the gradient of gray, which performs by fringes, makes it difficult for traditional algorithms to evaluate its definition. This paper proposes a method based on nonsubsampled contourlet transformation (NSCT) combined with fringe direction as weighting coefficients to solve the problem. First, the information about the directions of these fringes in interferograms was evaluated. Then, the NSCT method was applied to interferograms so that several sub-images in different scales and directions can be achieved. Based on these sub-images and directions information, an improved function for definition evaluation was conducted. Lastly, a series of simulations and experiments was carried out to verify its feasibility and accuracy. (C) 2021 Optical Society of America