Text summarization evaluation using semantic probability distributions

Most popular methods for evaluation of automatic summarized text content employ some protocol that requires gold standard summary, usually made by human, for validating the summarized text content based on some content comparison. These evaluation methods are however unable to function in case human-made summaries are not available, or improperly functioning when these summaries are in poor quality. In this paper, we proposed SESP, a novel evaluation method using content based approach. SESP applies advanced text tokenization methods and semantic based similarity metrics to generate semantic probability distributions of text contents. 'the probability distributions are then used for evaluating summarized text content given the original text document. We showed that SESP functions without a need for gold-standard summaries, but yet achieving better performance compared with the state of the art methods that require human-made summaries.