Enhancing Noisy Binary Search Efficiency through Deep Reinforcement Learning

Noisy binary search (NBS) aims to find the closest element to a target value within a sorted array through erroneous queries. In an ideal NBS environment where the error rate remains constant, and the costs of all queries are the same, the maximum likelihood estimation (MLE) procedure has been proven to be the optimal decision strategy. However, in some non-ideal NBS problems, both the error rates and the costs are dependent on the queries, and in some cases, finding the optimal decision strategies can be intractable. We propose to use deep reinforcement learning to approximate the optimal decision strategy in the NBS problem, in which an intelligent agent is used to interact with the NBS environment. A dueling double deep Q-network guides the agent to take action at each step, either to generate a query or to stop the search and predict the target value. An optimized policy will be derived by training the network in the NBS environment until convergence. By evaluating our proposed algorithm on a non-ideal NBS environment, visual field test, we show that the performance of our proposed algorithm surpasses baseline visual field testing algorithms by a large margin.