My Actions Speak Louder Than Your Words: When User Behavior Predicts Their Beliefs About Agents' Attributes

An implicit expectation of asking users to rate agents, such as an AI decision-aid, is that they will use only relevant information-ask them about an agent's benevolence, and they should consider whether or not it was kind. Behavioral science, however, suggests that people sometimes use irrelevant information. We identify an instance of this phenomenon, where users who experience better outcomes in a human-agent interaction systematically rated the agent as having better abilities, being more benevolent, and exhibiting greater integrity in a post hoc assessment than users who experienced worse outcomes-which were the result of their own behavior-with the same agent. Our analyses suggest the need for augmentation of models so they account for such biased perceptions as well as mechanisms so that agents can detect and even actively work to correct this and similar biases of users.