A Methodological Framework for Statistical Analysis of Social Text Streams

Social media are one of the main contributors of user generated content; providing vast amounts of data in daily basis, covering a wide range of topics, interests and events. In order to identify and link meaningful and relevant information, clustering algorithms have been used to partition the user generated content. We have identified though that these algorithms exhibit various shortcomings when they have to deal with social media textual information, which is dynamic and streaming in nature. Thus we explore the idea to estimate the algorithms' parameters based on observations on the clusters' properties' (like the centroid, shape and density) evolution. By experimenting with the clusters' properties, we propose a methodological framework that detects the evolution of the clusters' centroid, shape and density and explores their role in parameters' estimation.