Manufacturing part identification using computer vision and machine learning

1. A method for identifying a manufacturing part, the method comprising:

receiving an image comprising a manufacturing part from an augmented reality device;

determining a bounding perimeter encapsulating the manufacturing part;

receiving a prediction label and a confidence value for the manufacturing part from a neural network;

generating a message comprising the image, the prediction label, and the confidence value; and

providing the message to the augmented reality device.

2. The method of claim 1 , further comprising applying a color and magnitude of gradients threshold to the image such that a background of the image is mostly eliminated, and a contour of the manufacturing part is remaining.

3. The method of claim 2 , further comprising determining a contour outline of the manufacturing part based on the remaining contour of the manufacturing part.

4. The method of claim 1 , wherein the augmented reality device comprises a camera and a display having augmented reality capabilities, and wherein the augmented reality device is configured to be worn by a user.

5. The method of claim 1 , wherein receiving the image comprises receiving a video stream from the augmented reality device, and wherein the method further comprises extracting the image as a single frame from the video stream.

6. The method of claim 1 , further comprising extracting a sub-image comprising image data within confines of the bounding perimeter and resizing the sub-image to create a resized sub-image to be processed by the neural network.

7. The method of claim 1 , further comprising providing a manufacturing part picklist to the augmented reality device, wherein the manufacturing part picklist comprises data to guide a user operating the augmented reality device to retrieve one or more manufacturing parts.

8. The method of claim 1 , wherein the neural network comprises a convolutional neural network trained for object detection.

9. The method of claim 1 , wherein generating the message comprises overlaying the prediction label and the confidence value on the image such that the prediction label and the confidence value are located near or overlaid on top of the manufacturing part in the image.

10. A manufacturing part identification system comprising:

an augmented reality device comprising a camera;

a server in communication with the augmented reality device comprising computer readable storage media storing instructions that, when executed by one or more processors cause the one or more processors to:

receive an image comprising a manufacturing part from the augmented reality device;

determine a contour outline of the manufacturing part based on the image; and

determine a bounding perimeter encapsulating the manufacturing part; and

a neural network in communication with the server, wherein the neural network is configured to:

determine a prediction label for the manufacturing part; and

determine a confidence value comprising a statistical likelihood that the prediction label is correct.

11. The system of claim 10 , wherein the augmented reality device comprises a camera and a display having augmented reality capabilities, and wherein the augmented reality device is configured to be worn by a user.

12. The system of claim 10 , wherein the server comprising computer readable storage media further causes the one or more processors to determine a sub-image, wherein the sub-image comprises the image cropped to the bounding perimeter encapsulating the manufacturing part.

13. The system of claim 12 , wherein the server comprising computer readable storage media further causes the one or more processors to resize the sub-image to a shape and/or resolution accepted by the neural network.

14. The system of claim 13 , wherein the neural network is further configured to receive the sub-image from the server in one channel as a grayscale image.

15. The system of claim 10 , wherein the server comprising computer readable storage media further causes the one or more processors to apply a color and magnitude of gradients threshold to the image such that a background of the image is mostly eliminated, and a contour of the manufacturing part remains.

16. The system of claim 10 , wherein the neural network comprises a convolution neural network trained for object detection.

17. The system of claim 10 , wherein the augmented reality device is configured to provide a video stream to the server in real-time for detection of manufacturing parts in real-time.

18. The system of claim 10 , wherein the server comprising computer readable storage media further causes the one or more processors to overlay the prediction label and the confidence value on the image, and provide the image to the augmented reality device.

19. The system of claim 10 , wherein the server comprising computer readable storage media further causes the one or more processors to provide a manufacturing part picklist to the augmented reality device, wherein the manufacturing part picklist comprises data to guide a user operating the augmented reality device to retrieve and select one or more manufacturing parts.

20. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:

receive an image comprising a manufacturing part from an augmented reality device;

apply a color and magnitude of gradients threshold to the image such that a background of the image is mostly eliminated, and a contour of the manufacturing part remains;

determine a contour outline of the manufacturing part;

determine a bounding perimeter encapsulating the contour outline;

receive a prediction label and a confidence value for the manufacturing part from a convolutional deep neural network;

generate a message comprising the image, the prediction label, and the confidence value; and

provide the message to the augmented reality device.