Distributed synthesized association mining for big transactional data

被引:4
|
作者
Pal, Amrit [1 ,2 ]
Kumar, Manish [2 ]
机构
[1] GLA Univ, Dept Comp Engn & Applicat, Mathura, India
[2] Indian Inst Informat Technol Allahabad, Dept Informat Technol, Prayagraj, India
关键词
Big Data; HDFS; MapReduce; Apriori; frequent itemset; association rule; DATA SETS; RULES; PATTERNS;
D O I
10.1007/s12046-020-01380-8
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Data is increasing rapidly day by day along with the transactional database. Dividing this data and storing it in a distributed manner is an effective way for storage and retrieval. Mining such distributed data with minimum dependence between sub-problems is a crucial task. Finding frequent itemsets and corresponding association rules is a big challenge while considering the aggregation in a distributed environment. To overcome these challenges, we propose a distributed frequent itemset generation and association rule mining algorithm using MapReduce programming model. The proposed scheme generates frequent itemset and mine association rules using a synthesized distributed technique. The rules are mined in a distributed manner, and then weights are assigned to subsets of data and association rules. A proper mixture of association rules that are generated in distributed manner is done using a weighted approach. This paper presents a novel MapReduce-based synthesis approach, which can work well over a distributed storage of large amount of data.
引用
收藏
页数:13
相关论文
共 50 条
  • [11] A Study on Association Rule Mining of Darknet Big Data
    Ban, Tao
    Eto, Masashi
    Guo, Shanqing
    Inoue, Daisuke
    Nakao, Koji
    Huang, Runhe
    2015 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2015,
  • [12] Memory-optimized distributed utility mining for big data
    Kumar, Sunil
    Mohbey, Krishna Kumar
    JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES, 2022, 34 (08) : 6491 - 6503
  • [13] Distributed Adaptive Model Rules for Mining Big Data Streams
    Anh Thu Vu
    De Francisci Morales, Gianmarco
    Gama, Joao
    Bifet, Albert
    2014 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2014, : 345 - 353
  • [14] Parallel and distributed clustering framework for big spatial data mining
    Bendechache, Malika
    Tari, A-Kamel
    Kechadi, M-Tahar
    INTERNATIONAL JOURNAL OF PARALLEL EMERGENT AND DISTRIBUTED SYSTEMS, 2019, 34 (06) : 671 - 689
  • [15] Distributed Bayesian Matrix Decomposition for Big Data Mining and Clustering
    Zhang, Chihao
    Yang, Yang
    Zhou, Wei
    Zhang, Shihua
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2022, 34 (08) : 3701 - 3713
  • [16] Distributed Stochastic Aware Random Forests - Efficient Data Mining for Big Data
    Assuncao, Joaquim
    Fernandes, Paulo
    Lopes, Lucelene
    Normey, Silvio
    2013 IEEE INTERNATIONAL CONGRESS ON BIG DATA, 2013, : 425 - 426
  • [17] Distributed Data Association Rule Mining: Tools and Techniques
    Sethi, Manoj
    Jindal, Rajni
    PROCEEDINGS OF THE 10TH INDIACOM - 2016 3RD INTERNATIONAL CONFERENCE ON COMPUTING FOR SUSTAINABLE GLOBAL DEVELOPMENT, 2016, : 481 - 485
  • [18] Mining Algorithm for Association Rules in Big Data Based on Hadoop
    Fu, Chunhua
    Wang, Xiaojing
    Zhang, Lijun
    Qiao, Liying
    ADVANCES IN MATERIALS, MACHINERY, ELECTRONICS II, 2018, 1955
  • [19] Issues in Quantitative Association Rule Mining: A Big Data Perspective
    Adhikary, Dhrubajit
    Roy, Swarup
    PROCEEDINGS OF INTERNATIONAL CONFERENCE ON ICT FOR SUSTAINABLE DEVELOPMENT ICT4SD 2015, VOL 2, 2016, 409 : 377 - 385
  • [20] Neutrosophic Association Rule Mining Algorithm for Big Data Analysis
    Abdel-Basset, Mohamed
    Mohamed, Mai
    Smarandache, Florentin
    Chang, Victor
    SYMMETRY-BASEL, 2018, 10 (04):