A Virtual Boarding System of an Autonomous Vehicle for Investigating the Effect of an AR Display on Passenger Comfort

This paper introduces our attempt to investigate the relationship between an augmented reality display and the sense of comfort of a passenger on board an autonomous vehicle, using an omnidirectional video display system. An omnidirectional camera is located above the passenger seat of a moving vehicle and captures omnidirectional scenes from in and around the vehicle. A virtual environment, specifically a spherical CG model placed in a virtual space, is constructed in a computer, and a panoramic video generated from the omnidirectional video is mapped on to it. A virtual camera that captures the passenger's direction of view is set at the centre of the CG sphere. The orientation of the virtual camera is set in line with the orientation of the passenger's view, which is acquired by an inertia sensor in a head-mounted display (HMD), so that the passenger can look around at the omnidirectional scene. We control the vehicle speed in the virtual space by changing the video replay speed in our developed system. Using this system, we conducted experiments to investigate the relationship between an augmented reality display and a passenger's sense of comfort, by superimposing the virtual pattern (depicting the vehicle's direction) generated using vehicle information recorded along with the video recording.