Distributed synthesized association mining for big transactional data

被引:4
|
作者
Pal, Amrit [1 ,2 ]
Kumar, Manish [2 ]
机构
[1] GLA Univ, Dept Comp Engn & Applicat, Mathura, India
[2] Indian Inst Informat Technol Allahabad, Dept Informat Technol, Prayagraj, India
关键词
Big Data; HDFS; MapReduce; Apriori; frequent itemset; association rule; DATA SETS; RULES; PATTERNS;
D O I
10.1007/s12046-020-01380-8
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Data is increasing rapidly day by day along with the transactional database. Dividing this data and storing it in a distributed manner is an effective way for storage and retrieval. Mining such distributed data with minimum dependence between sub-problems is a crucial task. Finding frequent itemsets and corresponding association rules is a big challenge while considering the aggregation in a distributed environment. To overcome these challenges, we propose a distributed frequent itemset generation and association rule mining algorithm using MapReduce programming model. The proposed scheme generates frequent itemset and mine association rules using a synthesized distributed technique. The rules are mined in a distributed manner, and then weights are assigned to subsets of data and association rules. A proper mixture of association rules that are generated in distributed manner is done using a weighted approach. This paper presents a novel MapReduce-based synthesis approach, which can work well over a distributed storage of large amount of data.
引用
收藏
页数:13
相关论文
共 50 条
  • [21] A review on big data based parallel and distributed approaches of pattern mining
    Kumar, Sunil
    Mohbey, Krishna Kumar
    JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES, 2022, 34 (05) : 1639 - 1662
  • [22] A Distributed Method for Fast Mining Frequent Patterns From Big Data
    Huang, Peng-Yu
    Cheng, Wan-Shu
    Chen, Ju-Chin
    Chung, Wen-Yu
    Chen, Young-Lin
    Lin, Kawuu W.
    IEEE ACCESS, 2021, 9 : 135144 - 135159
  • [23] A Group Mining Method for Big Data on Distributed Vehicle Trajectories in WAN
    Yang, Jie
    Li, Xiaoping
    Wang, Dandan
    Wang, Jia
    INTERNATIONAL JOURNAL OF DISTRIBUTED SENSOR NETWORKS, 2015,
  • [24] Big data mining with parallel computing: A comparison of distributed and MapReduce methodologies
    Tsai, Chih-Fong
    Lin, Wei-Chao
    Ke, Shih-Wen
    JOURNAL OF SYSTEMS AND SOFTWARE, 2016, 122 : 83 - 92
  • [25] A Solution for Mining Big Data Based on Distributed Data Streams and Its Classifying Algorithms
    Mao, Guojun
    Qiao, Jiewei
    DATA MINING AND BIG DATA, DMBD 2017, 2017, 10387 : 263 - 271
  • [26] Association feature mining algorithm of web accessing data in big data environment
    Gong, Jing
    JOURNAL OF DISCRETE MATHEMATICAL SCIENCES & CRYPTOGRAPHY, 2018, 21 (02): : 333 - 337
  • [27] Data Mining with Big Data
    Sowmya, R.
    Suneetha, K. R.
    PROCEEDINGS OF 2017 11TH INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND CONTROL (ISCO 2017), 2017, : 246 - 250
  • [28] Data Mining with Big Data
    Wu, Xindong
    Zhu, Xingquan
    Wu, Gong-Qing
    Ding, Wei
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2014, 26 (01) : 97 - 107
  • [29] EAFIM: efficient apriori-based frequent itemset mining algorithm on Spark for big transactional data
    Raj, Shashi
    Ramesh, Dharavath
    Sreenu, M.
    Sethi, Krishan Kumar
    KNOWLEDGE AND INFORMATION SYSTEMS, 2020, 62 (09) : 3565 - 3583
  • [30] Safety warning of coal mining face based on big data association rule mining
    Meng, Fanqiang
    Li, Chunxia
    JOURNAL OF COMPUTATIONAL METHODS IN SCIENCES AND ENGINEERING, 2022, 22 (04) : 1035 - 1052