An allele-sharing, moment-based estimator of global, population-specific and population-pair FST under a general model of population structure

Being able to properly quantify genetic differentiation is key to understanding the evolutionary potential of a species. One central parameter in this context is F-ST, the mean coancestry within populations relative to the mean coancestry between populations. Researchers have been estimating F-ST globally or between pairs of populations for a long time. More recently, it has been proposed to estimate population-specific F-ST values, and population-pair mean relative coancestry. Here, we review the several definitions and estimation methods of F-ST, and stress that they provide values relative to a reference population. We show the good statistical properties of an allele-sharing, method of moments based estimator of F-ST (global, population-specific and population-pair) under a very general model of population structure. We point to the limitation of existing likelihood and Bayesian estimators when the populations are not independent. Last, we show that recent attempts to estimate absolute, rather than relative, mean coancestry fail to do so.