Granted Patent B2

US 12,447,687 · App. 18/452,914 · Granted Oct 21, 2025

Inventors: Vadim Pinskiy (Wayne, NJ); Matthew C. Putman (Brooklyn, NY); Damas Limoge (Brooklyn, NY); Aswin Raghav Nirmaleswaran (Brooklyn, NY)

Assignee: Nanotronics Imaging, Inc.

B29C64/393B22F10/30B22F10/85B22F12/90B29C64/209B33Y10/00B33Y50/02G06F18/2411G06F18/295G06N3/04G06V10/764G06V10/82G06V10/993B22F10/12B22F10/18B22F10/25B22F10/28

Abstract

Claims (66)

1. A method of training a reinforcement learning model for performing a corrective action in a manufacturing process executing in a manufacturing system, the method comprising:

receiving, by a computing system, an image of a specimen at a processing node in the manufacturing process;

detecting, by the computing system, an error in the specimen based on the image of the specimen;

determining, by the computing system using a reinforcement learning model, a change to a manufacturing parameter to correct the error based on a policy;

determining, by the computing system, state information of the specimen, the state information comprising a current action performed on the specimen and a previous action performed by the specimen at an upstream processing node in the manufacturing process;

generating, by the computing system, a quality metric for the specimen based on the state information;

generating, by the computing system, a reward corresponding to the state information;

comparing, by the computing system, an expected reward corresponding to the state information to the generated reward;

determining, by the computing system, that there is a deviation between the reward and the expected reward that exceeds a threshold amount; and

based on the determining, updating, by the computing system, the policy implemented by the reinforcement learning model.

2. The method of claim 1 , further comprising:

prepending the state information of the specimen with a last action performed on the specimen.

3. The method of claim 1 , wherein generating, by the computing system, the quality metric for the specimen based on the state information comprises:

determining a tensile strength of the specimen undergoing manufacturing.

4. The method of claim 3 , wherein generating, by the computing system, the reward corresponding to the state information comprises:

optimizing the tensile strength of the specimen undergoing manufacturing.

5. The method of claim 1 , wherein updating, by the computing system, the policy implemented by the reinforcement learning model comprises:

updating the policy based on a comparison between the expected reward and the generated reward.

6. The method of claim 1 , wherein determining, by the computing system using the reinforcement learning model, the change to the manufacturing parameter to correct the error based on the policy comprises:

determining a plurality of corrective actions to be performed to the specimen to correct the error based on the policy.

7. The method of claim 1 , further comprising:

deploying the reinforcement learning model in the manufacturing system once the reinforcement learning model obtains a threshold level of accuracy.

8. A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising:

receiving, by the computing system, an image of a specimen at a processing node in a manufacturing process executed across a manufacturing system;

detecting, by the computing system, an error in the specimen based on the image of the specimen;

determining, by the computing system using a reinforcement learning model, a change to a manufacturing parameter to correct the error based on a policy;

determining, by the computing system, state information of the specimen, the state information comprising a current action performed on the specimen and a previous action performed by the specimen at an upstream processing node in the manufacturing process;

generating, by the computing system, a quality metric for the specimen based on the state information;

generating, by the computing system, a reward corresponding to the state information;

comparing, by the computing system, an expected reward corresponding to the state information to the generated reward;

determining, by the computing system, that there is a deviation between the reward and the expected reward that exceeds a threshold amount; and

based on the determining, updating, by the computing system, the policy implemented by the reinforcement learning model.

9. The non-transitory computer readable medium of claim 8 , further comprising:

prepending the state information of the specimen with a last action performed on the specimen.

10. The non-transitory computer readable medium of claim 8 , wherein generating, by the computing system, the quality metric for the specimen based on the state information comprises:

determining a tensile strength of the specimen undergoing manufacturing.

11. The non-transitory computer readable medium of claim 10 , wherein generating, by the computing system, the reward corresponding to the state information comprises:

optimizing the tensile strength of the specimen undergoing manufacturing.

12. The non-transitory computer readable medium of claim 8 , wherein updating, by the computing system, the policy implemented by the reinforcement learning model comprises:

updating the policy based on a comparison between the expected reward and the generated reward.

13. The non-transitory computer readable medium of claim 8 , wherein determining, by the computing system using the reinforcement learning model, the change to the manufacturing parameter to correct the error based on the policy comprises:

determining a plurality of corrective actions to be performed to the specimen to correct the error based on the policy.

14. The non-transitory computer readable medium of claim 8 , further comprising:

deploying the reinforcement learning model in the manufacturing system once the reinforcement learning model obtains a threshold level of accuracy.

15. A system comprising:

a processor; and

a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform operations comprising:

receiving an image of a specimen at a processing node in a manufacturing process;

detecting an error in the specimen based on the image of the specimen;

determining, using a reinforcement learning model, a change to a manufacturing parameter to correct the error based on a policy;

determining state information of the specimen, the state information comprising a current action performed on the specimen and a previous action performed by the specimen at an upstream processing node in the manufacturing process;

generating a quality metric for the specimen based on the state information;

generating a reward corresponding to the state information;

comparing an expected reward corresponding to the state information to the generated reward;

determining that there is a deviation between the reward and the expected reward that exceeds a threshold amount; and

based on the determining, updating the policy implemented by the reinforcement learning model.

16. The system of claim 15 , wherein the operations further comprise:

prepending the state information of the specimen with a last action performed on the specimen.

17. The system of claim 15 , wherein generating the quality metric for the specimen based on the state information comprises:

determining a tensile strength of the specimen undergoing manufacturing.

18. The system of claim 17 , wherein generating the reward corresponding to the state information comprises:

optimizing the tensile strength of the specimen undergoing manufacturing.

19. The system of claim 15 , wherein updating the policy implemented by the reinforcement learning model comprises:

updating the policy based on a comparison between the expected reward and the generated reward.

20. The system of claim 15 , wherein determining, using the reinforcement learning model, the change to the manufacturing parameter to correct the error based on the policy comprises:

determining a plurality of corrective actions to be performed to the specimen to correct the error based on the policy.

Assignments (2)

Continuity (10)

References Cited (281)