A Q-learning Based Continuous Tuning of Fuzzy Wall Tracking without Exploration

A simple and easy to implement is proposed to address wall tracking task of an autonomous robot. The robot should navigate in unknown environments, find the nearest wall, and track it solely based on locally sensed data. The proposed method benefits from coupling fuzzy logic and Q-learning to meet requirements of autonomous navigations. The robot summerizes the obtained information from the world into a set of fuzzy states. For each fuzzy state, there are some suggested actions. States are related to their corresponding actions via simple fuzzy if-then rules, designed by human reasoning. The robot selects the most encouraged action for each state by Q-learning and through online experiences. The objective is to design a wall tracking algorithm which can efficiently adapt itself to different wall shapes in completely unknown environments. Q-learning is applied without any exploration phase, i.e. no training environment is considered. Experimental results on simulated Khepera robot validate that the proposed method efficiently deals with various wall contours from simple straight shape to complex concave, convex, or polygon shapes. The robot successfully keeps track of walls while staying within predefined margins.