Katibeh: A Persian news summarizer using the novel semi-supervised approach

Nowadays, text summarization is one of the most important active research fields in information retrieval. The most of the supervised extractive summarization systems utilize learning-to-rank methods to score sentences according to their importance. They need a high-quality comprehensive summarization corpus, which is labeled manually by human experts. Unfortunately, this sort of corpus is not available for most low-resource languages such as Persian. In this study, first of all, a comprehensive human-labeled summarization corpus (called Bistoon) collected by the crowdsourcing approach is introduced, and then a Persian summarizer based on a novel semi-supervised summarization approach, which is a combination of co-training and self-training, is presented to overcome the absence of sufficient data. During an iterative process, the proposed system is learned by Bistoon corpus and applied to unlabeled texts to generate the most confident summaries. These summaries are added to Bistoon for more iterations. During iterations, the training corpus is grown and the quality of the summarizer is simultaneously improved. The proposed system has been compared to other well-known Persian summarizers over the Pasokh and Bistoon standard test data sets. The evaluation results show the superiority of our methods in terms of precision, F-measure, Rouge metrics, and also human judgments.