Semiconductor inspection and metrology systems for distributing job among the CPUs or GPUs based on logical image processing boundaries

1. A system for scalable and flexible job distribution having a plurality of worker nodes coupled to a master node comprising:

the master node having at least one processor to run a master job manager, wherein the master job manager is configured to:

receive input image data from a semiconductor inspection tool or a semiconductor metrology tool, wherein the input image data is of a semiconductor wafer or reticle;

divide the input image data into at least a first job; and

distribute the first job to a first CPU worker node of the plurality of worker nodes;

the plurality of worker nodes including:

the first CPU worker node coupled with the master node, wherein the first CPU worker node includes two or more CPUs and two or more GPUs, wherein one of the CPUs in the first CPU worker node runs a worker job manager for the first CPU worker node, wherein the worker job manager includes a module with a deep learning model;

a second CPU worker node coupled with the master node that includes one or more CPUs without any GPU; and

at least one GPU worker node coupled with the master node that includes one or more GPUs without any CPU other than to run the worker job manager for the GPU worker node;

wherein the worker job manager of the first CPU worker node is configured to divide the first job into a plurality of tasks to be processed by the two or more CPUs or the two or more GPUs based on logical image processing boundaries, wherein the plurality of tasks include defect detection and defect classification;

wherein the deep learning model of the first CPU worker node is configured to:

determine whether each of the tasks is a CPU-bound task or a GPU-bound task based on the logical image processing boundaries, wherein the CPU-bound task is assigned to one of the two or more CPUs in the first CPU worker node instead of to one of the two or more GPUs in the first CPU worker node, and wherein the GPU-bound task is assigned to one of the two or more GPUs in the first CPU worker node instead of to one of the two or more CPUs in the first CPU worker node;

wherein the worker job manager of the first CPU worker node is further configured to:

dispatch the CPU-bound tasks for CPU-bound algorithm processes and the GPU-bound tasks for GPU-bound algorithm processes or for a GPU job manager; and

wherein the GPU job manager of the first CPU worker node is configured to:

queue in an input queue the input image data of at least some of the GPU-bound tasks for the GPU job manager to prioritize ahead of a later GPU bound-task; and

distribute the input image data of the at least some of the GPU-bound tasks in equal batches to the two or more GPUs of the first CPU worker node for processing such that completion time of the plurality of tasks is minimized.

2. The system of claim 1 , wherein there are more of the CPU than the GPU in one of the worker nodes.

3. The system of claim 1 , wherein there are more of the GPU than the CPU in one of the worker nodes.

4. The system of claim 1 , wherein the master node is further configured to divide the input image data into a second job and to distribute the second job to one of the plurality of worker nodes.

5. The system of claim 1 , wherein one of the CPU in the second CPU worker node runs the worker job manager for the second CPU worker node.

6. The system of claim 1 , further comprising an interface layer configured to communicate with an integrated memory controller (IMC) client using an application programming interface.

7. The system of claim 1 , further comprising a neural network to execute the deep learning model.

8. A method for scalable and flexible job distribution having a plurality of worker nodes coupled to a master node comprising:

receiving input image data from a semiconductor inspection tool or a semiconductor metrology tool at the master node, wherein the input image data is of a semiconductor wafer or reticle, and wherein the master node has a master job manager;

dividing the input image data into at least a first job using the master job manager;

distributing, using the master job manager, the first job to a first CPU worker node of the plurality of worker nodes coupled with the master node, wherein the plurality of worker nodes includes:

the first CPU worker node that includes two or more CPUs and two or more GPUs, wherein one of the CPUs in the first CPU worker node runs a worker job manager for the first CPU worker node, wherein the worker job manager includes a module with a deep learning model;

a second CPU worker node coupled with the master node that includes one or more CPUs without any GPU; and

at least one GPU worker node coupled with the master node that includes one or more GPUs without any CPU other than to run the worker job manager for the GPU worker node;

dividing, using the worker job manager of the first CPU worker node, the first job into a plurality of tasks to be processed by the two or more CPUs or the two or more GPUs based on logical image processing boundaries, wherein the plurality of tasks include defect detection and defect classification;

determining, using the deep learning model of the first CPU worker node, whether each of the tasks is a CPU-bound task or a GPU-bound task based on the logical image processing boundaries, wherein the CPU-bound task is assigned to one of the two or more CPUs in the first CPU worker node instead of to one of the two or more GPUs in the first CPU worker node, and wherein the GPU-bound task is assigned to one of the two or more GPUs in the first CPU worker node instead of to one of the two or more CPUs in the first CPU worker node;

dispatching, using the worker job manager of the first CPU worker node, the CPU-bound tasks for CPU-bound algorithm processes and the GPU-bound tasks for GPU-bound algorithm processes or for a GPU job manager;

queuing, using the GPU job manager of the first CPU worker node, in an input queue the input image data of at least some of the GPU-bound tasks for the GPU job manager to prioritize ahead of a later GPU bound-task; and

distributing, using the GPU job manager of the first CPU worker node, the input image data of the at least some of the GPU-bound tasks in equal batches to the two or more GPUs of the first CPU worker node for processing such that completion time of the plurality of tasks is minimized.

9. The method of claim 8 , wherein the method further comprises retraining the deep learning model.

10. The method of claim 8 , wherein the worker job managers operate under a first in first out job queue.

11. The method of claim 8 , wherein the input image data is from multiple wafer locations, and wherein the input image data is processed in a same batch.

12. The method of claim 8 , wherein the first job is distributed to the first CPU worker node in parallel and in real-time with other jobs from the input image data distributed to the plurality of worker nodes.

13. The method of claim 12 , wherein the first job is distributed to the first CPU worker node as the input image data is acquired in memory.