We evaluated the impact of visual similarity and action similarity on visual object identification. We taught participants to associate novel objects with nonword labels and verified that in memory visually similar objects were confused more often than visually dissimilar objects. We then taught participants to associate novel actions with nonword labels and verified that similar actions were confused more often than dissimilar actions. We then paired specific objects with specific actions. Visually similar objects paired with similar actions were confused more often in memory than when these same objects were paired with dissimilar actions. Hence the actions associated with objects served to increase or decrease their separation in memory space, and influenced the ease with which these objects could be identified. These experiments ultimately demonstrated that when identifying stationary objects, the memory of how these object were used dramatically influenced the ability to identify these objects.