Implementation and Validation of a Master-Slave Distributed Crawler Applied in Internet Information Retrieval

被引：0
|
作者：
Passos, Anderson [1 ]
机构：
[1] Miyazaki Int Coll, Dept Comparat Culture, Miyazaki, Japan
来源：
2012 7TH INTERNATIONAL CONFERENCE ON COMPUTING AND CONVERGENCE TECHNOLOGY (ICCCT2012) | 2012年
关键词：
web crawling; information retrieval; data mining; parallel execution; master-slave; distributed execution;
D O I：
暂无
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
This study describes a preliminary research on Information Retrieval (IR) systems. We developed a prototype tool that runs on a master-slave model and uses distributed processing in order to decentralize the workload while retrieving information from the Internet. Later, its viability is demonstrated with a set of executions and the discussion follows with an analysis of the master process overload versus the number of slaves connected.
引用
收藏
页码：870 / 875
页数：6
相关论文
共 50 条
  • [31] Robust multiestimation based adaptive control of robotic manipulators applied to master-slave robotic tandems
    Ibeas, A
    de la Sen, M
    2004 IEEE CONFERENCE ON ROBOTICS, AUTOMATION AND MECHATRONICS, VOLS 1 AND 2, 2004, : 31 - +
  • [32] Adaptive noise canceller with master-slave structure for information transmission systems in the electrified railway
    Kang, S.L.
    Qiu, Z.D.
    Xiao, Y.
    Tiedao Xuebao/Journal of the China Railway Society, 2001, 23 (02):
  • [33] Master-Slave principle applied to an electrically-tunable swept source-OCT system
    Marques, Manuel J.
    Cernat, Ramona
    Ensher, Jason
    Bradu, Adrian
    Podoleanu, Adrian
    OPTICAL COHERENCE TOMOGRAPHY AND COHERENCE DOMAIN OPTICAL METHODS IN BIOMEDICINE XXIV, 2020, 11228
  • [34] Research on master-slave distributed large-scale poultry farming measurement and control system
    Qun, Yu
    Zhang, Yan
    Wang, Xiuli
    Zhou, Zhunan
    Xian, Pingping
    Zhang, Fenghang
    2019 INTERNATIONAL CONFERENCE ON INTERNET OF THINGS (ITHINGS) AND IEEE GREEN COMPUTING AND COMMUNICATIONS (GREENCOM) AND IEEE CYBER, PHYSICAL AND SOCIAL COMPUTING (CPSCOM) AND IEEE SMART DATA (SMARTDATA), 2019, : 552 - 557
  • [35] Hierarchically partitioned coordinated operation of distributed integrated energy system based on a master-slave game
    Li, Peng
    Wang, Zixuan
    Yang, Weihong
    Liu, Haitao
    Yin, Yunxing
    Wang, Jiahao
    Guo, Tianyu
    ENERGY, 2021, 214
  • [36] Distributed communication interference resource scheduling using the master-slave parallel scheduling genetic algorithm
    Wei, Zhenhua
    Wu, Wenpeng
    Zhan, Jianwei
    Zhang, Zhaoguang
    SCIENTIFIC REPORTS, 2025, 15 (01):
  • [37] Modular DC Circuit Breaker With Master-Slave Concept for Gate Driver Simplification: Topology and Implementation
    Ren, Yu
    Zhang, Fan
    Yang, Xu
    Han, Xiaoqing
    Chen, Wenjie
    IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2022, 69 (09) : 8915 - 8925
  • [38] Design and Implementation of Distributed Information Retrieval System
    Liu, Haishu
    FIFTH INTERNATIONAL CONFERENCE ON DIGITAL IMAGE PROCESSING (ICDIP 2013), 2013, 8878
  • [39] An Internet of Things (IoT)-Based Master-Slave Regionalized Intelligent LED-Light-Controlling System
    Lee, Chun-Te
    Chen, Liang-Bi
    Chu, Huan-Mei
    Hsieh, Che-Jen
    Liang, Wei-Chieh
    APPLIED SCIENCES-BASEL, 2022, 12 (01):
  • [40] Investigation of navigation information correction techniques for master-slave AUV formations in unstable communication environments
    Wang, Qiuying
    Cai, Muchun
    Guo, Zheng
    Zhou, Wenhao
    MEASUREMENT, 2024, 229
12345