SOMMA: Cortically Inspired Paradigms for Multimodal Processing

SOMMA (Self Organizing Maps for Multimodal Association) consists on cortically inspired paradigms for multimodal data processing. SOMMA defines generic cortical maps one for each modality -composed of 3-layers cortical columns. Each column learns a discrimination to a stimulus of the input flow with the BCMu learning rule [26]. These discriminations are self-organized in each map thanks to the coupling with neural fields used as a neighborhood function [25]. Learning and computation in each map is influenced by other modalities thanks to bidirectional topographic connections between all maps. This multimodal influence drives a joint self-organization of maps and multimodal perceptions of stimuli. This work takes place after the design of a self-organizing map [25] and of a modulation mechanism for influencing its self-organization [26] oriented towards a multimodal purpose. In this paper, we introduce a way to connect these self-organizing maps to obtain a multimap multimodal processing, completing our previous work. We also give an overview of the architectural and functional properties of the resulting paradigm SOMMA.