Evaluating HDR Rendering Algorithms

A series of three experiments has been performed to test both the preference and accuracy of high dynamic-range (HDR) rendering algorithms in digital photography application. The goal was to develop a methodology for testing a wide variety of previously published tone-mapping algorithms for overall preference and rendering accuracy. A number of algorithms were chosen and evaluated first in a paired-comparison experiment for overall image preference. A rating-scale experiment was then designed for further investigation of individual image attributes that make up overall image preference. This was designed to identify the correlations between image attributes and the overall preference results obtained from the first experiments. In a third experiment, three real-world scenes with a diversity of dynamic range and spatial configuration were designed and captured to evaluate seven HDR rendering algorithms for both of their preference and accuracy performance by comparing the appearance of the physical scenes and the corresponding tone-mapped images directly. In this series of experiments, a modified Durand and Dorsey's bilateral filter technique consistently performed well for both preference and accuracy, suggesting that it is a good candidate for a common algorithm that could be included in future HDR algorithm testing evaluations. The results of these experiments provide insight for understanding of perceptual HDR image rendering and should aid in design strategies for spatial processing and tone mapping. The results indicate ways to improve and design more robust rendering algorithms for general HDR scenes in the future. Moreover, the purpose of this research was not simply to find out the "best" algorithms, but rather to find a more general psychophysical experiment based methodology to evaluate HDR image-rendering algorithms. This paper provides an overview of the many issues involved in an experimental framework that can be used for these evaluations.