MultiAICL: Multi-task Tuning for Augmented In-Context Learning in Text Style Transfer

In-context learning (ICL) enhances the performance of large language models (LLMs) across various natural language process (NLP) tasks by simply demonstrating a few-shot of examples or instructions during inference. However, ICL still encounters significant challenges on the text style transfer (TST) tasks, which require high levels of model reasoning. The existing ICL ability has not been further developed because LLMs lack the process of training and learning in context. To address these issues, we introduce Multi-Task Tuning for Augmented In-Context Learning (MultiAICL), a framework designed to enhance model ICL ability by simulating LLM's supervised fine-tuning steps. MultiAICL contains three main components: firstly, we construct example instructions for multiple tasks from the text corpus, where these examples are in the form of text-label pairs; secondly, we propose the Multi-Task Tuning (MTT) module, which tunes the model by randomly combining example instructions; and thirdly, we design the Augmented In-context Learning (AICL) module, which incorporates different tasks into example templates to infer the model. MultiAICL improves the ICL ability of LLMs while maintaining their generalization across multiple tasks, thus encouraging models to generate high-quality text. Extensive experiments show that MultiAICL achieves excellent results on all 6 TST tasks, even outperforming larger LLMs. The code and data are available at https://github.com/fuz999/NLPCC-2024- MultiAICL.