Implementation and Validation of a Master-Slave Distributed Crawler Applied in Internet Information Retrieval

被引：0
|
作者：
Passos, Anderson [1 ]
机构：
[1] Miyazaki Int Coll, Dept Comparat Culture, Miyazaki, Japan
来源：
2012 7TH INTERNATIONAL CONFERENCE ON COMPUTING AND CONVERGENCE TECHNOLOGY (ICCCT2012) | 2012年
关键词：
web crawling; information retrieval; data mining; parallel execution; master-slave; distributed execution;
D O I：
暂无
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
This study describes a preliminary research on Information Retrieval (IR) systems. We developed a prototype tool that runs on a master-slave model and uses distributed processing in order to decentralize the workload while retrieving information from the Internet. Later, its viability is demonstrated with a set of executions and the discussion follows with an analysis of the master process overload versus the number of slaves connected.
引用
收藏
页码：870 / 875
页数：6
相关论文
共 50 条
  • [1] Supporting internet protocols in master-slave fieldbus networks'
    Pacheco, F
    Tovar, E
    Kalogeras, A
    Pereira, N
    FIELDBUS SYSTEMS AND THEIR APPLICATIONOS 2001 (FET'2001), 2002, : 285 - 291
  • [2] Distributed support vector machine in master-slave mode
    Chen, Qingguo
    Cao, Feilong
    NEURAL NETWORKS, 2018, 101 : 94 - 100
  • [3] Master-slave algebra: On the binary compositions of distributed beliefs
    Adamatzky, A
    APPLIED MATHEMATICS AND COMPUTATION, 1998, 95 (2-3) : 173 - 180
  • [4] Designing and realizing of a master-slave distributed CNC system
    Hangkong Zhizao Gongcheng, 6 (32-34):
  • [5] Analysis of a master-slave architecture for distributed evolutionary computations
    Dubreuil, M
    Gagné, C
    Parizeau, M
    IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS, 2006, 36 (01): : 229 - 235
  • [6] Master-slave algebra: on the binary compositions of distributed beliefs
    Appl Math Comput (New York), 2-3 (173-180):
  • [7] Formal Verification of Distributed Master-Slave Finite State Machine
    Popovic, Marko
    Marinkovic, Vladimir
    Djukic, Miodrag
    Popovic, Miroslav
    2021 29TH TELECOMMUNICATIONS FORUM (TELFOR), 2021,
  • [8] An efficient multilevel master-slave model for distributed parallel computation
    Chen, HC
    Lim, A
    Warsi, NA
    BOUNDARY ELEMENT TECHNOLOGY XIII: INCORPORATING COMPUTATIONAL METHODS AND TESTING FOR ENGINEERING INTEGRITY, 1999, 2 : 439 - 448
  • [9] Research and Implementation of Master-Slave Control for Interventional Robot System
    Xie Qingcheng
    Chen Guo
    Han Shipeng
    2017 2ND ASIA-PACIFIC CONFERENCE ON INTELLIGENT ROBOT SYSTEMS (ACIRS), 2017, : 234 - 239
  • [10] The Design and Implementation of MPI Master-Slave Parallel Genetic Algorithm
    Liu, Shuping
    Cheng, Yanliu
    INTERNATIONAL CONFERENCE ON GRAPHIC AND IMAGE PROCESSING (ICGIP 2012), 2013, 8768
12345