Assessing the performance of neural encoding models in the presence of noise

An analytical method is introduced for evaluating the performance of neural encoding models. The method addresses a critical question that arises during the course of the development and validation of encoding models: is a given model near optimal in terms of its accuracy in predicting the stimulus-elicited responses of a neural system, or can the predictive accuracy be improved significantly by further model development? The evaluation method is based on a derivation of the minimum mean-square error between actual responses and modeled responses. It is formulated as a comparison between the mean-square error of the candidate model and the theoretical minimum mean-square error attainable through an optimal model for the system. However, no a priori information about the nature of the optimal model is required. The theoretically minimum error is determined solely from the coherence function between pairs of system responses to repeated presentations of the same dynamic stimulus. Thus, the performance of the candidate model is judged against the performance of an optimal model rather than against that of an arbitrarily assumed model. Using this method, we evaluated a linear model for neural encoding by mechanosensory cells in the cricket cereal system. At low stimulus intensities, the best-fit linear model of encoding by single cells was found to be nearly optimal, even though the coherence between stimulus-response pairs (a commonly used measure of system linearity) was low. In this low-stimulus-intensity regime, the mean square error of the linear model was on the order of the power of the cell responses. In contrast, at higher stimulus intensities the linear model was not an accurate representation of neural encoding, even though the stimulus-response coherence was substantially higher than in the low-intensity regime.