Towards unifying pre-trained language models for semantic text exchange

In the field of Social Internet of Things (SIoT), the widespread adoption of Internet of Things (IoT) devices and the rise of social media have resulted in a significant amount of social IoT data, including textual data. However, the mining and analysis of these data have been a challenging task. To address this issue, we leverage semantic text exchange techniques from the field of Natural Language Processing (NLP) to generate text that preserves the original style while modifying its semantic information. this task can be used for a wide range of NLP tasks, including data augmentation, adversarial attacks on text, and conversational systems. However, the present methods of semantic text exchange have some shortcomings, such as the generated text is not natural, the content is not rich, and the cost of model training is high. This paper presents a semantically independent sentence generation method, which can generate fluent and diverse semantically independent sentences. We combine three word substitution patterns with a pre-trained text infilling model to generate text. The essential advantage of the proposed method is that it does not require training the model for text infilling on specific datasets, which greatly reduces the cost of text generation. Experimental results show that our method generates more fluent and more diverse sentences than baselines. And it has a good performance in sentiment preservation of the original sentence and semantic similarity between the replacement words and the generated sentences.