Hierarchical reinforcement learning for chip-macro placement in integrated circuit

The complexity of chip design has consistently grown, adhering to Moore's law. In this paper, we examine a crucial step in integrated circuit design called chip macro placement. Traditionally, human experts are consulted to optimize placement for reduced power consumption, but this requires significant effort. Recently, machine learning -based methods have emerged to address this task, showing promising results. Our work aims to tackle key bottlenecks in current reinforcement learning (RL) based methods, which often suffer from sparse rewards, large state-action search spaces, and unstable training, resulting in suboptimal solutions. To address these issues, we propose a novel approach called Hierarchical Reinforcement Learning for Placement (HRLP). This method distinguishes itself in two ways. First, we introduce a hierarchical RL framework that learns human expert design patterns by assigning agent -specific placement sub -tasks (referred to as options in reinforcement learning) instead of primitive actions at each time step within an episode. Second, we generate dense rewards for each episode by calculating the difference in local wirelength and congestion. We conducted experiments on eight public chip design benchmarks, adaptec and bigblue series. Our model has an average improvement of 9.29 % and 14.25 % on Half Perimeter Wire Length (HPWL), the huge improvement shows that our method is capable of converging to a better sub -optimal than DeepPlace and providing a promising future for AI to aid the design of circuit placement.