Bilateral Contract Prices Estimation using a Q-Learning based approach

The electricity markets restructuring process encouraged the use of computational tools in order to allow the study of different market mechanisms and the relationships between the participating entities. Automated negotiation plays a crucial role in the decision support for energy transactions due to the constant need for players to engage in bilateral negotiations. This paper proposes a methodology to estimate bilateral contract prices, which is essential to support market players in their decisions, enabling adequate risk management of the negotiation process. The proposed approach uses an adaptation of the QL-earning reinforcement learning algorithm to choose the best from a set of possible contract prices forecasts that are determined using several methods, such as artificial neural networks (ANN), support vector machines (SVM), among others. The learning process assesses the probability of success of each forecasting method, by comparing the expected negotiation price with the historic data contracts of competitor players. The negotiation scenario identified as the most probable scenario that the player will face during the negotiation process is the one that presents the higher expected utility value. This approach allows the supported player to be prepared for the negotiation scenario that is the most likely to represent a reliable approximation of the actual negotiation environment.