Design of Hadoop-based Framework for Analytics of Large Synchrophasor Datasets

The power sector is increasingly utilizing GPS-stamped real-time measurements from Phasor Measurement Units (PMU) to improve the reliability and efficiency of power grids. PMUs directly measure phase angles in real-time, which allows operators to perform grid optimization that was not possible in the past. In early 2010, almost 250 PMU's were deployed across North America and it continues to increase remarkably. However, one of the major challenges is the complexity of analyzing such a large amount of real-time datasets. The phasor data from PMU's will be accumulated in peta-bytes in coming years, which exceeds the capability of conventional relational database technologies. A new software and architecture framework is in desperate need to process such a large amount of data in real-time reliably and cost-effectively. The paper presents a new framework based on Hadoop, an open-source system widely used in the industry, to perform distributed and parallel analytics on large synchrophasor datasets. The paper demonstrates various applications of MapReduce to analyze patterns of load distribution using parallel node calculations, which can later be scaled up to match the requirements for power utility sector. The paper serves as a pilot study on data analytics on big data of smart grids.