COMET: Client-Oriented METadata Service for Highly Available Distributed File Systems

被引：1
|
作者：
Xue, Ruini [1 ]
Ao, Lixiang [1 ]
Guan, Zhongyang [1 ]
机构：
[1] Univ Elect Sci & Technol China, Sch Engn & Comp Sci, Chengdu, Peoples R China
来源：
2015 27TH INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE AND HIGH PERFORMANCE COMPUTING (SBAC-PAD) | 2015年
关键词：
Distributed Storage; File System; Metadata; High Availability; Hadoop; HDFS; MAPREDUCE;
D O I：
10.1109/SBAC-PAD.2015.27
中图分类号：
TP3 [计算技术、计算机技术];
学科分类号：
0812 ;
摘要：
Highly available metadata services of distributed file systems are essential to cloud applications. However, existing highly available metadata designs lack client-oriented features that treat metadata discriminately, leading to a single metadata fault domain and low availability. After investigating the workload characteristics of Hadoop, we propose Client-Oriented METadata (COMET), a novel highly available metadata service design that divides and distributes metadata into independent regions in terms of clients. These regions are isolated fault domains inherently, and failures in one region will not break file operations in other regions. A prototype of COMET was implemented based on HDFS, and the experimental results show that COMET can significantly improve metadata availability of HDFS without obvious performance degradation. It can also deliver scalable performance and faster metadata recovery due to its decentralized architecture.
引用
收藏
页码：154 / 161
页数：8
相关论文
共 50 条
  • [1] A Flattened Metadata Service for Distributed File Systems
    Li, Siyang
    Liu, Fenlin
    Shu, Jiwu
    Lu, Youyou
    Li, Tao
    Hu, Yang
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2018, 29 (12) : 2641 - 2657
  • [2] A Highly Reliable Metadata Service for Large-Scale Distributed File Systems
    Zhou, Jiang
    Chen, Yong
    Wang, Weiping
    He, Shuibing
    Meng, Dan
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2020, 31 (02) : 374 - 392
  • [3] A Distributed Cache Framework for Metadata Service of Distributed File Systems
    Sun, Yao
    Liu, Jie
    Ye, Dan
    Zhong, Hua
    2013 19TH IEEE INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED SYSTEMS (ICPADS 2013), 2013, : 51 - 58
  • [4] A client-oriented distributed panorama simulation system
    Guo, YC
    DCABES and ICPACE Joint Conference on Distributed Algorithms for Science and Engineering, 2005, : 195 - 200
  • [5] LocoFS: A Loosely-Coupled Metadata Service for Distributed File Systems
    Li, Siyang
    Lu, Youyou
    Shu, Jiwu
    Hu, Yang
    Li, Tao
    SC'17: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS, 2017,
  • [6] Towards a Client-Oriented Model of Types and States in Service-Oriented Development
    Atkinson, Colin
    Bostan, Philipp
    EDOC: 2009 IEEE INTERNATIONAL ENTERPRISE DISTRIBUTED OBJECT COMPUTING CONFERENCE, 2009, : 119 - 127
  • [7] A Low-Latency Metadata Service for Geo-Distributed File Systems
    Lin, Chuangwei
    Liu, Bowen
    Zhou, Wei
    Xu, Yueyue
    Zhang, Xuyun
    Dou, Wanchun
    WEB INFORMATION SYSTEMS ENGINEERING - WISE 2021, PT I, 2021, 13080 : 87 - 100
  • [8] MetaFlow: A Scalable Metadata Lookup Service for Distributed File Systems in Data Centers
    Sun, Peng
    Wen, Yonggang
    Duong Nguyen Binh Ta
    Xie, Haiyong
    IEEE TRANSACTIONS ON BIG DATA, 2018, 4 (02) : 203 - 216
  • [9] COMET: A Distributed Metadata Service for Federated Cloud Infrastructures
    Wang, Cong
    Thareja, Komal
    Stealey, Michael
    Ruth, Paul
    Baldin, Ilya
    2019 IEEE HIGH PERFORMANCE EXTREME COMPUTING CONFERENCE (HPEC), 2019,
  • [10] Analyzing Metadata Performance in Distributed File Systems
    Biardzki, Christoph
    Ludwig, Thomas
    PARALLEL COMPUTING TECHNOLOGIES, PROCEEDINGS, 2009, 5698 : 8 - +
12345