Assembly error correction for assembly lines
Aspects of the disclosed technology provide a computational model that utilizes machine learning for detecting errors during a manual assembly process and determining a sequence of steps to complete the manual assembly process in order to mitigate the detected errors. In some implementations, the disclosed technology evaluates a target object at a step of an assembly process where an error is detected to a nominal object to obtain a comparison. Based on this comparison, a sequence of steps for completion of the assembly process of the target object is obtained. The assembly instructions for creating the target object are adjusted based on this sequence of steps.
1. A method for optimizing workflow in an assembly line, the method comprising:
monitoring tooling at a step of an assembly process for assembling a target object;
based on the monitoring, detecting, at the step of the assembly process of the target object, an error in assembly of the target object using a first neural network model;
evaluating data associated with the tooling at the step of the assembly process using a second neural network to predict a quality metric for the target object;
comparing the quality metric for the target object at the step of the assembly process to a nominal object at the step of the assembly process;
based on the comparing, determining that the quality metric is out of specification;
based the determining, generating an updated sequence of steps to bring the target object into specification using a gated recurrent unit model; and
adjusting subsequent steps in the assembly process using updated assembly instructions for the target object based on the updated sequence of steps.
2. The method of claim 1 , wherein the target object is evaluated against the nominal object at the step of the assembly process.
3. The method of claim 1 , wherein the target object is evaluated against a final configuration of the nominal object.
4. The method of claim 1 , wherein the updated sequence of steps is configured to minimize a deviation between the target object and the nominal object.
5. The method of claim 4 , wherein the deviation is determined based on a similarity between the updated sequence of steps to complete the assembly process of the target object and another sequence of steps to complete the assembly process of the nominal object.
6. The method of claim 4 , wherein the deviation is minimized using a Markov Decision Process (MDP) through a reward formulation.
7. The method of claim 1 , wherein a stochastic gradient descent method is used to derive the updated sequence of steps to complete the assembly process of the target object.
8. A system for optimizing workflow in an assembly line, the system comprising:
a plurality of image capture devices, wherein each of the plurality of image capture devices is disposed at a different position to capture movement of an operator during an assembly process of a target object; and
an assembly instruction module configured to automatically modify guidance and instructions provided to the operator, wherein the assembly instruction module is coupled to the plurality of image capture devices, and wherein the assembly instruction module is configured to perform operations comprising:
receiving, by the assembly instruction module, motion data from the plurality of image capture devices, wherein the motion data corresponds to performance of a set of steps by the operator to assemble the target object;
determining, based on the motion data and at a step of the set of steps, an error in assembly of the target object using a first neural network model;
evaluating the motion data using a second neural network model to predict a quality metric for the target object;
comparing the quality metric for the target object to a nominal object;
based on the comparing, determining that the quality metric is out of specification;
based the determining, generating an updated sequence of steps to bring the target object into specification using a gated recurrent unit model; and
adjusting subsequent steps in the assembly processing using updated assembly instructions that are provided to the operator based on the updated sequence of steps.
9. The system of claim 8 , wherein the motion data includes digital recordings of hand movements of the operator during the assembly of the target object.
10. The system of claim 8 , wherein the assembly instruction module is further configured to apply a stochastic gradient descent method to derive the updated sequence of steps.
11. The system of claim 8 , wherein the updated sequence of steps is configured to minimize a deviation between the target object and the nominal object.
12. The system of claim 11 , wherein the deviation is determined based on a similarity between the updated sequence of steps to complete the assembly of the target object and another sequence of steps to complete the assembly of the nominal object.
13. The system of claim 11 , wherein the deviation is minimized using a Markov Decision Process (MDP) through a reward formulation.
14. The system of claim 8 , wherein the assembly instruction module is further configured to:
extract, from the motion data, a set of images that represent the assembly of the target object; and
evaluate the set of images to identify the performance of the set of steps by the operator to assemble the target object.
15. A non-transitory computer-readable media comprising instructions stored thereon which, when executed by one or more processors, are configured to cause the one or more processors to perform operations comprising:
monitoring tooling at a step of an assembly process for assembling a target object;
based on the monitoring, detecting, at the step of the assembly process of a target object, an error in assembly of the target object using a first neural network model;
evaluating data associated with the tooling at the step of the assembly process using a second neural network to predict a quality metric for the target object;
comparing the quality metric for the target object at the step of the assembly process to a nominal object at the step of the assembly process;
based on the comparing, determining that the quality metric is out of specification;
based the determining, generating an updated sequence of steps to bring the target object into specification using a gated recurrent unit model; and
adjusting subsequent steps in the assembly process using updated assembly instructions for the target object based on the updated sequence of steps.
16. The non-transitory computer-readable media of claim 15 , further comprising:
deriving, using a stochastic gradience descent method, the updated sequence of steps.
17. The non-transitory computer-readable media of claim 15 , wherein the target object is evaluated against the nominal object at the step of the assembly process.
18. The non-transitory computer-readable media of claim 15 , wherein the target object is evaluated against a final configuration of the nominal object.
19. The non-transitory computer-readable media of claim 15 , wherein the updated sequence of steps is configured to minimize a deviation between the target object and the nominal object.
20. The non-transitory computer-readable media of claim 19 , wherein the deviation is minimized using a Markov Decision Process (MDP) through a reward formulation.