The simplex gradient and noisy optimization problems

Many classes of methods for noisy optimization problems are based on function information computed on sequences of simplices. The Nelder-Mead, multidirectional search, and implicit filtering methods are three such methods. The performance of these methods can be explained in terms of the difference approximation of the gradient implicit in the function evaluations. Insight can be gained into choice of termination criteria, detection of failure, and design of new methods.