Humans are able to distinguish eye movement-induced retinal image motion and physical object motion during smooth pursuit eye movements. We investigated the neurophysiological basis of this ability by comparing motion-onset visual evoked potentials (VEPs) to onset of: (1) physical object motion during fixation, (2) eye movement-induced retinal image motion, and (3) physical object motion during eye movements. Electro-oculographic (EOG) artifacts were removed and the influence of eye-movement quality was evaluated. Retinal image shift was of similar magnitude in all conditions (9degrees/s) and elicited typical motion-onset VEPs, with N2 at occipital and P2 at central derivations. During smooth pursuit, physical object motion induced N2 and P2 of higher latencies than during fixation. In the absence of physical object motion, i.e., for exclusively eye movement-induced retinal image motion, the N2 amplitude was reduced. This is taken as evidence that the activity of detectors of physical object motion is reflected by a part of the N2 component. N2 also reflects eye movement-induced retinal image motion. It is concluded that headcentric motion detection and the detection of eye movement-induced retinal image motion is mediated by brain mechanisms with similar latencies and, within the resolution limits of VEPs, at similar locations.