Motion Understanding: Task-Directed Attention and Representations that Link Perception with Action

This short paper outlines my position on a future direction for computational research on visual motion understanding. The direction combines motion perception, visual attention, action representation and computational vision. Due the breadth of literature in these areas, the paper cannot present a comprehensive review of any one topic. The review is a selective one, a selection that attempts to make some particular points. I claim that task-directed attentive processing is a largely unexplored dimension in the computational motion field. I recount in the context of motion understanding a past argument that in order to make vision systems general, attention is one of the components of the strategy. No matter how sophisticated the methods become for extracting motion information from image sequences, it will not be possible to achieve the goal of human-like performance without integrating the optimization of processing that attention provides. Virtually all past surveys of computational models of motion processing completely ignore attention. However, the concept has crept into work over the years in a variety of ways. A second claim is that the biology of attention offers some interesting insights to guide future development. Many computational authors had previously commented that too little is known about how biological vision systems use task-directed attention in motion processing; this is no longer true. Here, I briefly summarize biological evidence that attentive processing affects all aspects of visual perception including motion, and again emphasize that this paper does not do justice to the breadth and depth of the field. New findings provide a critical link between the perception of visual actions and their execution. Together these findings point to a strategy for motion understanding closely related to that presented more than two decades ago.