From Perceptrons to Deep Neural Networks

Deep neural networks are intensively researched field of artificial intelligence. Big companies like Google, Microsoft, Baidu or Facebook are supporting research and development in this field. The recent victory over human player in the game of Go points to a huge potential of this approach. Machine learning approaches based on deep learning techniques bring significant gain over existing methods based on manually tuned features in different areas. In this paper we present the evolution of deep neural networks from first neuron models towards today's deep architectures.