Interactive mining of multimedia databases with virtual reality

We present in this paper a new 3D interactive method for visualizing multimedia data with virtual reality. We consider that a domain expert has collected a set of examples described with numeric and symbolic attributes but also with sounds, images, videos or Web sites, and that this expert wishes to explore those data in order to understand their structure. We use a 3D stereoscopic display in order to let the expert easily visualize and perceive the numerical attributes. We add to this display the appearing of contextual information such as texts, voice synthesis, sound, small images. Larger images, videos and web sites are displayed on a second computer in order to ensure real time display. Navigating through the data is done in a very intuitive and precise way with a 3D sensor with six degrees of freedom that simulates a virtual camera. Interactive request can be formulated by the expert with a data glove that recognize the gestures. We show how this tool has been applied to a real world application in the context of human skin analysis.