E2RLIXT: An end-to-end framework for robust index tuning based on reinforcement learning

Index selection is crucial for improving database query performance, and yet it remains a challenging problem. Recent work has explored using Reinforcement Learning (RL) to address this problem by formulating it as a decision problem where an agent learns to recommend indexes. However, existing approaches have not thoroughly investigated the formulation and representation of this index selection problem (ISP) in the context of RL, nor have they addressed the adaptation to highly recurrent workloads common in real-world systems. We propose E2RLIXT, an End-to-End RL-based robust IndeX Tuning framework, to address these gaps. Within this framework, we design a unified strategy for representing both single- and multi-column indexes, explore two state representation strategies, and employ a reward design that considers index interactions without biasing the agent's learning. We employ Proximal Policy Optimization with data augmentation for stable training and design a rollout algorithm to enhance the agent's ability to adapt to varied workloads sharing common query templates. To the best of our knowledge, we are the first to design and integrate rollout algorithms into RL-based ISP solutions. Experimental results show that our solutions outperform comparative approaches and provide robust performance across diverse workloads.