Query Processing on Large Graphs: Scalability Through Partitioning

Graphs, as an expressive data structure, have become increasingly important for modeling real-world applications (collaboration, different kinds of transactions, social networks, to name a few.) With the advent of social networks and the web, the graphs have grown too large to fit in main memory. This calls for alternative approaches, algorithms, and their analysis to develop an efficient, scalable evaluation of queries on graphs of any size. In this paper, we use the time-tested "divide and conquer" approach by partitioning a graph into desired number of partitions and process queries over those partitions to obtain all or specified number of answers. This entails correctly computing answers that span multiple partitions or need the same partition more than once. A query evaluation approach along with the necessary minimal book keeping is proposed and its correctness established. Query answering on partitioned graphs also requires analyzing partitioning schemes for their impact on query processing and determining the number as well as the sequence in which partitions are loaded to reduce the response time for processing one or a batch of queries. We correlate query properties and partition characteristics to reduce query processing time in terms of the number of partitions loaded. We identify a set of quantitative metrics and use them for formulating heuristics to determine the order of loading partitions for efficient query processing. Extensive experiments on large graphs (synthetic and real-world) using different partitioning schemes analyze the proposed heuristics on a variety of query types. An existing graph querying system has been extended to evaluate queries on partitioned graphs.