Automated Workload Management Using Machine Learning

Mainframe System processing includes a "Batch Cycle" that approximately spans 8 pm to 8 am, every week, from Monday night to Saturday morning. The core part of the cycle completes around 2 am, with key client deliverables associated with the end times of certain jobs, tracked by Service Delivery. There are single and multi-client batch streams, a QA stream which includes all clients, and about 2,00,000 batch jobs per day that execute. Despite a sophisticated job scheduling software, and automated system workload management, operator intervention is required, or believed to be required, to reprioritize when and what jobs get available system resources. Our work is to characterize, analyse and visualize the reasons for a manual change in the schedule. The work requires extensive data preprocessing and building machine learning models for the causal relationship between various system variables and the time of manual changes.