Draw & Dance: Voice-controlled AR Assistant

We introduce an Augmented Reality (AR) assistant that reacts to your voice, music and sound. The user can draw and place stick figures in the environment that will dance to the music and respond to specific commands. The stick figure characters can also augment a Google Home by taking its place on top of the speaker and moving in response to the rhythm of the music. They will also change their accessories and clothing depending on the genre of the music, play instruments or change their dancing style. We explore how voice UI may benefit from a visual component that re-enforces a mental model for otherwise structureless interactions. The lack of a visual representation of voice assistants can make the adoption and learning curve slower. Our system can help to discover capabilities of a physical tethered device that otherwise would be imperceptible. We present the first system that embodies a voice-controlled speaker using an AR smart avatar.