Real-time Image Based Lighting for outdoor Augmented Reality under dynamically changing illumination conditions

Knowledge about illumination conditions in a real world scene has many applications, among them Augmented Reality which aims at placing virtual objects in the real world. An important factor for convincing augmentations is to use the illumination of the real world when rendering the virtual objects so they are shaded consistently and cast consistent shadows. The work presented in this paper aims at making a robust system capable of estimating the lighting of an outdoor scene, and apply the light changes to the virtual augmented objects that are placed within a real scene. The method uses an Irradiance Volume, modified to use an environment map of a given scene, to mimic the multiple lights reflected in a scene using Image Based Lighting, while normal Phong shading is used to mimic the sun shading. These are combined with a Shadow Volume method to ensure shadow interaction with the surrounding environment. For every frame an Illumination Estimation approximates local illumination light parameters used in the rendering of the augmented objects. The light parameters are furthermore used to, at runtime, create new environment maps, to update the irradiance volume. The result is a rendering pipeline capable of handling dynamic light changes, and applies them to augmented objects within a given scene, enabling realistic augmentations under changing illumination conditions.