Cue combination and color edge detection in natural scenes

Biological vision systems are adept at combining cues to maximize the reliability of object boundary detection, but given a set of co-localized edge detectors operating on different sensory channels, how should their responses be combined to compute overall edge probability? To approach this question, we collected joint responses of red-green and blue-yellow edge detectors both ON- and OFF-edges using a human-labeled image database as ground truth (D. Martin, C. Fowlkes, D. Tal, & J. Malik, 2001). From a Bayesian perspective, the rule for combining edge cues is linear in the individual cue strengths when the ON-edge and OFF-edge joint distributions are (1) statistically independent and (2) lie in an exponential ratio to each other. Neither condition held in the color edge data we collected, and the function P(ON vertical bar cues)-dubbed the "combination rule"-was correspondingly complex and nonlinear. To characterize the statistical dependencies between edge cues, we developed a generative model ("saturated common factor," SCF) that provided good fits to the measured ON-edge and OFF-edge joint distributions. We also found that a divisive normalization scheme derived from the SCF model transformed raw edge detector responses into values with simpler distributions that satisfied both preconditions for a linear combination rule. A comparison to another normalization scheme (O. Schwartz & E. Simoncelli, 2001) suggests that apparently minor details of the normalization process can strongly influence its performance. Implications of the SCF normalization scheme for cue combination in biological sensory systems are discussed.