Overall Writing Effectiveness: Exploring Students' Use of LLMs, Pushing the Limits of Automated Text Generation

The advent of generative artificial intelligence for text generation, epitomized by the introduction of ChatGPT in November 2022, represents a significant shift in the academic writing paradigm. This pre-study examines how students make use of Large Language Models (LLMs) for their academic writing processes, transitioning from solitary writing to true human-machine collaboration. Participants were recruited from a workshop on LLMs and were subsequently interviewed qualitatively after two weeks of unsupervised usage. These interviews were designed using the new Overall Writing Effectiveness (OWE) framework and focused on LLMs' role in academic writing. The qualitative content of these interviews was analysed following Mayring's methodology. Findings indicate that LLMs did not substantially accelerate the writing process but enhanced the quality of the texts and redefined writing as a collaborative effort. This study not only explores the limits of automation in academic writing but also highlights how generative AI is pushing the boundaries of what is considered genuine human capabilities. This analysis opens the discussion of how to incorporate such technologies into future education curriculums.