MapReduce: A Flexible Data Processing Tool

MapReduce (MR) has emerged as a flexible data processing tool for different applications. It has emerged as a programming model for processing and generating large data sets. Users specify a map function that processes a key or value pair to generate a set of intermediate key or value pairs and a reduce function that merges all intermediate values associated with the same intermediate key. MapReduce automatically parallelizes and executes the program on a large cluster of commodity machines. The runtime system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing required inter-machine communication. It also allows new programmers with parallel and distributed systems to easily utilize the resources of a large distributed system.