Performance of visually guided tasks using simulated prosthetic vision and saliency-based cues

Objective. The objective of this paper is to evaluate the benefits provided by a saliency-based cueing algorithm to normally sighted volunteers performing mobility and search tasks using simulated prosthetic vision. Approach. Human subjects performed mobility and search tasks using simulated prosthetic vision. A saliency algorithm based on primate vision was used to detect regions of interest (ROI) in an image. Subjects were cued to look toward the directions of these ROI using visual cues superimposed on the simulated prosthetic vision. Mobility tasks required the subjects to navigate through a corridor, avoid obstacles and locate a target at the end of the course. Two search task experiments involved finding objects on a tabletop under different conditions. Subjects were required to perform tasks with and without any help from cues. Results. Head movements, time to task completion and number of errors were all significantly reduced in search tasks when subjects used the cueing algorithm. For the mobility task, head movements and number of contacts with objects were significantly reduced when subjects used cues, whereas time was significantly reduced when no cues were used. The most significant benefit from cues appears to be in search tasks and when navigating unfamiliar environments. Significance. The results from the study show that visually impaired people and retinal prosthesis implantees may benefit from computer vision algorithms that detect important objects in their environment, particularly when they are in a new environment.