3D object rotation-based mechanical parts selection through 2D image processing
View Patent ↗Technologies are generally described for 3D object recognition through 2D image processing based on white balancing and object-rotation in machine vision systems. According to some examples, image recognition of an object captured with a camera under insufficient lighting may be achieved through white balancing. Processing cost reduction may be achieved in the learning process for image recognition through automatic generation of rotated 2D images of target objects to be detected, such as machine parts, from a small number of 2D images of a target object and generation of a 3D image of the target object from the rotated 2D images. Image recognition may thus be ensured even under insufficient lighting through execution of the image recognition process for multiple images and learning the successful recognition results. Some examples may be implemented in mechanical parts selection, where 2D images of the parts may be available beforehand.
1. A method for 3D object recognition through rotation-based 2D image processing, the method comprising:
obtaining a plurality of images of an object, wherein each image corresponds to a distinct white balance value and a rotation state of the object;
storing the plurality of images along with an image identifier for each image, a parts identifier common to one or more images, and a weight value for each image, wherein the parts identifier indicates images that are associated with a single object;
in response to a request from an image recognition module, providing one or more images of the object to the image recognition module such that the object is recognized by the image recognition module based on a comparison of the provided one or more images to a captured image of the object;
determining a depth and a portion of the object closest to an image capture device by extracting outlines of portions of the object based on a reflectivity of the portions;
combining the portion of the object closest to the image capture device with the plurality of images of the object, wherein the plurality of images depict the object at a plurality of rotational angles; and
recognizing the 3D object based on the combination and the determination.
2. The method of claim 1 , wherein obtaining the plurality of images of the object comprises:
capturing the plurality of images at distinct exposure settings and for distinct rotation states of the object.
3. The method of claim 1 , wherein obtaining the plurality of images of the object comprises:
receiving an image of the object; and
generating the plurality of images of the object from the received image using a plurality of white balance values and a plurality of rotation states.
4. The method of claim 3 , wherein generating the plurality of images of the object comprises:
varying a white balance value of the received image incrementally for each of the plurality of images.
5. The method of claim 3 , wherein generating the plurality of images of the object from the received image using the plurality of rotation states comprises:
detecting a central symmetry axis of the object based on one or more symmetrical features of the object; and
generating the plurality of images of the object by expanding and contracting the received image around the central symmetry axis and a second axis.
6. The method of claim 5 , wherein the second axis is substantially perpendicular to the central symmetry axis.
7. The method of claim 1 , further comprising:
assigning weight values to the stored plurality of images.
8. The method of claim 7 , wherein assigning weight values to the stored plurality of images comprises:
assigning an initial weight value to each stored image; and
incrementing a weight value of an image that is matched to the captured image of the object by the image recognition module.
9. The method of claim 1 , further comprising:
reducing a noise caused by reflection from other objects in each of the plurality of images of the object.
10. A method for 3D object recognition through rotation-based 2D image processing, the method comprising:
receiving a captured image of a target object;
receiving a plurality of images from an image database, along with an image identifier for each image, a parts identifier common to one or more images, and a weight value for each image, wherein the parts identifier indicates images that are associated with an object, and wherein each image corresponds to a distinct white balance value and a rotation state of the target object;
comparing the plurality of images to the captured image of the target object;
in response to a determination that one of the plurality of images matches the captured image of the target object, recognizing the target object as the object associated with the one of the plurality of images;
determining a depth and a portion of the target object closest to an image capture device by extracting outlines of portions of the target object based on a reflectivity of the portions;
combining the portion of the target object closest to the image capture device with the plurality of images of the target object, wherein the plurality of images depict the target object at a plurality of rotational angles; and
recognizing the 3D object based on the combination and the determination.
11. The method of claim 10 , further comprising:
in response to a determination that none of the plurality of images matches the captured image of the target object, adjusting one or more of a white balance value and a rotation state of the captured image of the target object; and
comparing the plurality of images to the adjusted captured image of the target object.
12. The method of claim 10 , further comprising:
providing feedback associated with the determination to a detection control module configured to manage a database that stores the plurality of images such that a weight value of the one of the plurality of images is updated at the database.
13. The method of claim 10 , further comprising:
iteratively eliminating inconsistencies due to one or more of a tight reflection, a rotation, and a center point or a line of the target object.
14. A system for 3D object rotation-based mechanical parts selection through 2D image processing, the system comprising:
a computer processor;
a non-transitory computer readable memory connected to the computer processor, the non-transitory computer readable memory storing a detection control module, wherein the detection control module is configured to:
obtain a plurality of images of an object, wherein each image corresponds to a distinct white balance value and a rotation state of the object;
store the plurality of images along with an image identifier for each image, a parts identifier common to one or more images, and a weight value for each image, wherein the parts identifier indicates images that are associated with a single object; and
in response to a request from an image recognition module, provide one or more images of the object to the image recognition module such that the object is recognized by the image recognition module based on a comparison of the provided one or more images to a captured image of the object; and
a parts rotation module configured to:
determine a depth and a portion of the object closest to an image capture device by extracting outlines of portions of the object based on a reflectivity of the portions; and
combine the portion of the object closest to the image capture device with the plurality of images of the object, wherein the plurality of images depict the object at a plurality of rotational angles; and
recognize the 3D object based on the combination and the determination.
15. The system of claim 14 , wherein the detection control module is configured to obtain the plurality of images of the object through capture of the plurality of images with an image capture module at distinct exposure settings and for distinct rotation states of the object.
16. The system of claim 14 , wherein the detection control module is configured to obtain the plurality of images of the object by receiving a 2D image of the object and generating the plurality of images of the object from the received image using a plurality of white balance values and a plurality of rotation states.
17. The system of claim 16 , further comprising:
the non-transitory computer readable memory storing a white balance adjustment module, wherein the white balance adjustment module is configured to:
vary a white balance value of the received image incrementally for each of the plurality of images.
18. The system of claim 14 , further comprising:
the non-transitory computer readable memory storing the parts rotation module, wherein the parts rotation module is configured to:
adjust a rotation state of the object in each image by:
detecting a central symmetry axis of the object based on one or more symmetrical features of the object; and
generating the plurality of images of the object by the expanding and contracting the received image around the central symmetry axis and a second axis.
19. The system of claim 18 , wherein the second axis is substantially perpendicular to the central symmetry axis.
20. The system of claim 14 , wherein the detection control module is further configured to:
assign weight values to the stored plurality of images.
21. The system of claim 20 , wherein the detection control module is configured to assign weight values to the stored plurality of images by:
assigning an initial weight value to each stored image; and
incrementing a weight value of an image that is matched to the captured image of the object by the image recognition module.
22. A system for 3D object rotation-based mechanical parts selection through 2D image processing, the system comprising:
a computer processor;
a non-transitory computer readable memory connected to the computer processor, the non-transitory computer readable memory storing a detection control module, wherein the
detection control module is configured to:
obtain a plurality of images of an object, wherein each image corresponds to a distinct white balance value and a rotation state of the object;
store the plurality of images along with an image identifier for each image, a parts identifier common to one or more images, and a weight value for each image, wherein the parts identifier indicates images that are associated with a single object; and
in response to a request from an image recognition module, provide one or more images of the object to the image recognition module such that the object is recognized by the image recognition module based on a comparison of the provided one or more images to a captured image of the object;
an image capture device configured to:
capture an image of a target object;
the object recognition module configured to:
receive the one or more of the plurality of images from the detection control module;
compare the one or more of the plurality of images to the captured image of the target object; and
in response to a determination that one of the plurality of images matches the captured image of the target object, recognize the target object as the object associated with the one of the plurality of images; and
the non-transitory computer readable memory storing a parts rotation module is further configured to:
determine a depth and a portion of the object closest to an image capture device by extracting outlines of portions of the object based on a reflectivity of the portions; and combine the portion of the object closest to the image capture device with the plurality of images of the object, wherein the plurality of images depict the object at a plurality of rotational angles.
23. The system of claim 22 , further comprising:
the non-transitory computer readable memory storing a white balance adjustment module, wherein the white balance adjustment module is configured to:
vary a white balance value of the received image incrementally for each of the plurality of images; and
further wherein the non-transitory computer readable memory storing the parts rotation module, wherein the parts rotation module is configured to:
adjust a rotation state of the object in each image by:
detecting a central symmetry axis of the object based on one or more symmetrical features of the object; and
generating the plurality of images of the object by the expanding and contracting the received image around the central symmetry axis and a second axis.
24. The system of claim 22 , wherein the system is integrated within a parts picker system.