HaoLap: A Hadoop based OLAP system for big data

被引：30
|
作者：
Song, Jie [1 ]
Guo, Chaopeng [1 ]
Wang, Zhi [1 ]
Zhang, Yichan [1 ]
Yu, Ge [2 ]
Pierson, Jean-Marc [3 ]
机构：
[1] Northeastern Univ, Software Coll, Shenyang 110819, Peoples R China
[2] Northeastern Univ, Sch Informat & Engn, Shenyang 110819, Peoples R China
[3] Univ Toulouse 3, Lab IRIT, F-31062 Toulouse, France
来源：
JOURNAL OF SYSTEMS AND SOFTWARE | 2015年 / 102卷
基金：
新加坡国家研究基金会; 中国国家自然科学基金;
关键词：
Cloud data warehouse; Multidimensional data model; MapReduce;
D O I：
10.1016/j.jss.2014.09.024
中图分类号：
TP31 [计算机软件];
学科分类号：
081202 ; 0835 ;
摘要：
In recent years, facing information explosion, industry and academia have adopted distributed file system and MapReduce programming model to address new challenges the big data has brought. Based on these technologies, this paper presents HaoLap (Hadoop based oLap), an OLAP (OnLine Analytical Processing) system for big data. Drawing on the experience of Multidimensional OLAP (MOLAP), HaoLap adopts the specified multidimensional model to map the dimensions and the measures; the dimension coding and traverse algorithm to achieve the roll up operation on dimension hierarchy; the partition and linearization algorithm to store dimensions and measures; the chunk selection algorithm to optimize OLAP performance; and MapReduce to execute OLAP. The paper illustrates the key techniques of HaoLap including system architecture, dimension definition, dimension coding and traversing, partition, data storage, OLAP and data loading algorithm. We evaluated HaoLap on a real application and compared it with Hive, HadoopDB, HBaseLattice, and Olap4Cloud. The experiment results show that HaoLap boost the efficiency of data loading, and has a great advantage in the OLAP performance of the data set size and query complexity, and meanwhile HaoLap also completely support dimension operations. (C) 2014 Elsevier Inc. All rights reserved.
引用
收藏
页码：167 / 181
页数：15
相关论文
共 50 条
  • [31] Hadoop as Big Data Operating System - The Emerging Approach for Managing Challenges of Enterprise Big Data Platform
    Mazumdar, Sourav
    Dhar, Subhankar
    2015 IEEE FIRST INTERNATIONAL CONFERENCE ON BIG DATA COMPUTING SERVICE AND APPLICATIONS (BIGDATASERVICE 2015), 2015, : 499 - 504
  • [32] Mining Algorithm for Association Rules in Big Data Based on Hadoop
    Fu, Chunhua
    Wang, Xiaojing
    Zhang, Lijun
    Qiao, Liying
    ADVANCES IN MATERIALS, MACHINERY, ELECTRONICS II, 2018, 1955
  • [33] Computation services and applications of electricity big data based on hadoop
    Wang, Xiangwei
    Shi, Yuliang
    Zhang, Jianlin
    Liang, Bo
    Cheng, Cuiping
    Dianwang Jishu/Power System Technology, 2015, 39 (11): : 3128 - 3133
  • [34] Research On Big Data Information Retrieval Based on Hadoop Architecture
    Chen Jie
    Chen Dongjie
    Huang Bangming
    2014 IEEE WORKSHOP ON ELECTRONICS, COMPUTER AND APPLICATIONS, 2014, : 492 - 495
  • [35] Hadoop-Based Big Data Distributions: A Comparative Study
    Hamdaoui, Ikram
    El Fissaoui, Mohamed
    El Makkaoui, Khalid
    El Allali, Zakaria
    EMERGING TRENDS IN INTELLIGENT SYSTEMS & NETWORK SECURITY, 2023, 147 : 242 - 252
  • [36] Parallel Implementation of PrePost Algorithm Based on Hadoop for Big Data
    Rochd, Yassir
    Hafidi, Imad
    2018 IEEE 5TH INTERNATIONAL CONGRESS ON INFORMATION SCIENCE AND TECHNOLOGY (IEEE CIST'18), 2018, : 24 - 28
  • [37] Hadoop-Based Intelligent Care System (HICS): Analytical Approach for Big Data in IoT
    Rathore, M. Mazhar
    Paul, Anand
    Ahmad, Awais
    Anisetti, Marco
    Jeon, Gwanggil
    ACM TRANSACTIONS ON INTERNET TECHNOLOGY, 2017, 18 (01)
  • [38] A Semantically-Based Big Data Processing System Using Hadoop and Map-Reduce
    Wang Wanting
    Qin Zheng
    SOCIALLY AWARE ORGANISATIONS AND TECHNOLOGIES: IMPACT AND CHALLENGES, 2016, 477 : 246 - 247
  • [39] Research on Hadoop Platform Big Data Software System Based on Forest Random Search Algorithm
    Li, Yujun
    Lecture Notes in Electrical Engineering, 2022, 935 LNEE : 653 - 659
  • [40] Advances in data warehousing and OLAP in the big Data Era
    Bellatreche, Ladjel
    Cuzzocrea, Alfredo
    Song, Il-Yeol
    INFORMATION SYSTEMS, 2015, 53 : 39 - 40
12345