Balancing Efficiency and Performance in NLP: A Cross-Comparison of Shallow Machine Learning and Large Language Models via AutoML

This study critically examines the resource efficiency and performance ofShallow Machine Learning (SML) methods versus Large Language Models (LLMs)in text classification tasks by exploring the balance between accuracy and environ-mental sustainability. We introduce a novel optimization strategy that prioritizescomputational efficiency and ecological impact alongside traditional performancemetrics leveraging Automated Machine Learning (AutoML). Our analysis revealsthat while the pipelines we developed did not surpass state-of-the-art (SOTA) modelsregarding raw performance, they offer a significantly reduced carbon footprint. Wediscovered SML optimal pipelines with competitive performance and up to 70 timesless carbon emissions than hybrid or fully LLM pipelines, such as standard BERT andDistilBERT variants. Similarly, we obtain hybrid pipelines (using SML and LLMs)with between 20% and 50% reduced carbon emissions compared to fine-tuned alter-natives and only a marginal decrease in performance. This research challenges theprevailing reliance on computationally intensive LLMs for NLP tasks and underscoresthe untapped potential of AutoML in sculpting the next wave of environmentallyconscious AI models.