Generating integrated circuit placements using neural networks
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.
1. A method of training a node placement neural network that comprises:
an encoder neural network that is configured to, at each of a plurality of time steps, receive an input representation comprising data representing a current state of a placement of a netlist of nodes on a surface of an integrated circuit chip as of the time step and process the input representation to generate an encoder output, and
a policy neural network configured to, at each of the plurality of time steps, receive an encoded representation generated from the encoder output generated by the encoder neural network and process the encoded representation to generate a score distribution over a plurality of positions on the surface of the integrated circuit chip, the method comprising:
generating a reinforcement learning training example, comprising:
obtaining training netlist data specifying a training netlist of nodes;
generating a training placement of the training netlist of nodes using the node placement neural network, and
determining a value of a reward function that measures a quality of the training placement of the training netlist of nodes, wherein the reward function comprises a plurality of terms that each measure a respective characteristic of the training placement; and
training the policy neural network on the reinforcement learning training example through reinforcement learning.
2. The method of claim 1 , wherein training the policy neural network through reinforcement learning comprises holding values of parameters of the encoder neural network fixed during the training of the policy neural network through reinforcement learning.
3. The method of claim 2 , further comprising:
prior to training the policy neural network through reinforcement learning, training the encoder neural network through supervised learning.
4. The method of claim 1 , wherein the reward function includes a wire length term that measures a wire length of wires on the surface of the integrated circuit chip.
5. The method of claim 1 , wherein the reward function includes a timing term that measures a timing performance of the integrated circuit chip.
6. The method of claim 1 , wherein the reward function includes a power term that measures a power or energy consumed by the integrated circuit chip.
7. The method of claim 1 , wherein the reward function includes a power term that measures an area consumed by the training placement.
8. The method of claim 1 , wherein the reward function includes a congestion term that measures congestion on a surface of the integrated circuit chip.
9. The method of claim 1 , wherein the reward function is a sum or a weighted sum of the plurality of terms.
10. The method of claim 1 , further comprising:
after training the node placement neural network:
receiving new netlist data;
fine-tuning the trained node placement neural network on the new netlist data through reinforcement learning; and
generating an integrated circuit placement for the new netlist data using the fine-tuned node placement neural network, comprising placing a respective node from the new netlist data at each of a plurality of time steps using score distributions generated by the fine-tuned node placement neural network.
11. A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a node placement neural network that comprises:
an encoder neural network that is configured to, at each of a plurality of time steps, receive an input representation comprising data representing a current state of a placement of a netlist of nodes on a surface of an integrated circuit chip as of the time step and process the input representation to generate an encoder output, and
a policy neural network configured to, at each of the plurality of time steps, receive an encoded representation generated from the encoder output generated by the encoder neural network and process the encoded representation to generate a score distribution over a plurality of positions on the surface of the integrated circuit chip, the operations comprising:
generating a reinforcement learning training example, comprising:
obtaining training netlist data specifying a training netlist of nodes;
generating a training placement of the training netlist of nodes using the node placement neural network, and
determining a value of a reward function that measures a quality of the training placement of the training netlist of nodes, wherein the reward function comprises a plurality of terms that each measure a respective characteristic of the training placement; and
training the policy neural network on the reinforcement learning training example through reinforcement learning.
12. The system of claim 11 , wherein training the policy neural network through reinforcement learning comprises holding values of parameters of the encoder neural network fixed during the training of the policy neural network through reinforcement learning.
13. The system of claim 12 , the operations further comprising:
prior to training the policy neural network through reinforcement learning, training the encoder neural network through supervised learning.
14. The system of claim 11 , wherein the reward function includes a wire length term that measures a wire length of wires on the surface of the integrated circuit chip.
15. The system of claim 11 , wherein the reward function includes a timing term that measures a timing performance of the integrated circuit chip.
16. The system of claim 11 , wherein the reward function includes a power term that measures a power or energy consumed by the integrated circuit chip.
17. The system of claim 11 , wherein the reward function includes a power term that measures an area consumed by the training placement.
18. The system of claim 11 , wherein the reward function is a sum or a weighted sum of the plurality of terms.
19. The system of claim 11 , wherein the reward function includes a congestion term that measures congestion on a surface of the integrated circuit chip.
20. One or more non-transitory computer-readable media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a node placement neural network that comprises:
an encoder neural network that is configured to, at each of a plurality of time steps, receive an input representation comprising data representing a current state of a placement of a netlist of nodes on a surface of an integrated circuit chip as of the time step and process the input representation to generate an encoder output, and
a policy neural network configured to, at each of the plurality of time steps, receive an encoded representation generated from the encoder output generated by the encoder neural network and process the encoded representation to generate a score distribution over a plurality of positions on the surface of the integrated circuit chip, the operations comprising:
generating a reinforcement learning training example, comprising:
obtaining training netlist data specifying a training netlist of nodes;
generating a training placement of the training netlist of nodes using the node placement neural network, and
determining a value of a reward function that measures a quality of the training placement of the training netlist of nodes, wherein the reward function comprises a plurality of terms that each measure a respective characteristic of the training placement; and
training the policy neural network on the reinforcement learning training example through reinforcement learning.