Architecture of Distributed Data Aggregation Service

The ever-growing trend of deploying applications over the Internet has resulted in increasingly tougher constraints and requirements. Data management systems are a major concern when it comes to scalability, flexibility and reliability due to being implemented in a distributed way. In this paper we present a Distributed Data Aggregation Service relying on a storage system designed to meet these demands, namely BlobSeer. The primary goal is to serve as a repository backend for complex analysis and automatic mining of scientific data (like bibtex entries). Several requirements, derived from this objective, match BlobSeer's features: versioning used for lock-free access to data and different granularity of read / write operations. We proposed a model to perform the correct translation between BlobSeer's unstructured view of data and the user's structured view. We implemented a client providing a formal description for the data retrieval queries and a specification for a search API. A benchmark tool relying on a performance model of BlobSeer, will be used to automatically determine the best BlobSeer deployment configuration for a specific data aggregation pattern.