Synthesis of Complex Sound Scenes with Transformation of Recorded Spatial Sound in Virtual Reality

This paper presents a real-time method for automatic listener-centric transformation of spatial sound recordings allowing the listener to move in the virtual reality in relation to the recording. This enables sound designers to efficiently implement perceptually plausible complex sound scenes from such recordings. The method decomposes the omnidirectional signal of a B-format recording into granular virtual omnidirectional sources and then places them on an arbitrary surface defining a convex hull. The placement is based on an energetic analysis performed on the B-format signal. This synthesis is complemented with a separate diffuse synthesis that ideally reproduces the ambience and reverberation from the original signal. Listening experiments show that the method is plausible and the quality of the method is very good when the listener is inside the specific cylinder-shaped surface.