System and method of object detection using AI deep learning models
A system for object detection of a manufactured part. The system comprises a system controller electronically coupled to an image acquisition device and delivery mechanism. An electronically stored ordered object detection map is provided which comprises predetermined detectable objects associated with the manufactured part. The ordered object detection map is generated from output created by execution of a trained object detection model and by processing such output according to predetermined calibration criteria. The system causing a visual image of the manufactured part to be captured with image data being extracted and processed to render at least one of a pass determination and a fail determination. The system being configured to process the manufactured based upon the rendered pass/fail determination.
1. A system for object detection for an item of manufacture comprising:
an image capture device;
a fixture mechanism that is adapted to hold the item of manufacture in operative relation to the image capture device;
a system controller electronically coupled to the image acquisition device, the system controller having a programmable processor utilizing processing circuitry, electronic memory, a programmable logic controller and input/output circuitry, the system controller being configured to carry out a visual inspection of the item of manufacture that is held by the fixture mechanism, such inspection comprising detection of visually observable feature on the item of manufacture, said visually observable features comprising at least one of visual characteristics of the plurality of predetermined detectable objects, a predetermined order of the plurality of predetermined detectable objects relative to one another and a predetermined location of the plurality of predetermined detectable objects along the item of manufacture;
an ordered object detection map stored in the electronic memory, the ordered object detection map having reference data for the item of manufacture, the reference data comprising the visually observable feature of the item of manufacture, the ordered object detection map being generated from object detection map output processed according to predetermined calibration criteria, the object detection map output being created by execution of a trained object detection model on input comprising at least training image data taken from an image of reference items of manufacture and a label map comprising identifications of the plurality of predetermined detectable objects;
computer-readable instructions executable by the programmable logic controller, execution of said computer-readable instructions causing electronic signals to be sent between the system controller, the fixture mechanism and the image capture device, said electronic signals comprising at least first, second and third control signals;
wherein the first control signal is sent by the programable logic controller to the fixture mechanism to indicate that the item of manufacture is held by the fixture mechanism in operative relation to the image capture device;
image data from the visual image being identified and processed by the programable processor, the image data comprising at least one detectable objects that correspond with the visually observable features of the predetermined detectable objects, detectable objects differ from the visually observable features of the predetermined detectable objects and an absence of detectable objects from the visually observable features of the predetermined detectable objects, the detectable objects from the visual image being processed in relation to the reference data of the ordered object detection map to render at least one of a pass determination and a fail determination, wherein the pass determination is rendered where detectable objects from the visual image correspond to predetermined detectable objects of the reference data of the ordered object detection map and the fail determination is rendered where detectable objects from the visual image fail to correspond to predetermined detectable objects of the reference data of the ordered object detection map;
wherein the third control signal is sent by the programable logic controller to the fixture mechanism to indicate that the item of manufacture held by the fixture mechanism is ready to be transported to at least one of a first location and a second location, the first location being designated for deposit of the item of manufacture held by the fixture mechanism where a pass determination is rendered and the second location being designated for deposit of the item of manufacture held by the fixture mechanism where the fail determination is rendered.
2. The system of claim 1 where the ordered object detection map further comprises at least one of a calibrated order and position associated with each of the plurality of predetermined detectable objects and the image data further comprises at least one of a calibrated order and position associated with the predetermined detectable objects from the visual image.
3. The system of claim 1 further comprising at least one additional cycle of detection of the item of manufacture held by the fixture mechanism wherein a fourth control signal is sent by the programmable logic controller to the fixture mechanism to reposition the item of manufacture held by the fixture mechanism in a reoriented position relative the image capture device and a fifth control signal is sent by the programable logic controller to the image capture device to actuate the image capture device to capture a new visual image of the item of manufacture held by the fixture mechanism in the reoriented position, the image data further comprising detectable objects from the new visual image, wherein the pass determination is rendered where detectable objects from the new visual image correspond to predetermined detectable objects of the reference data of the ordered object detection map and the fail determination is rendered where detectable objects from the new visual image fail to correspond to predetermined detectable objects of the reference data of the ordered object detection map.
4. The system of claim 1 wherein the detection of the item of manufacture held by the fixture mechanism further comprises a vision analysis, the vision analysis being carried out separate from the acquisition of the visual image.
5. The system of claim 1 wherein the ordered object detection map is modified to mask at least one object of the plurality of detectable objects by application of reference indicia to at least a portion of the training image data such that objects within the reference indicia are not recognizable upon being processed to render at least one of the pass determination and the fail determination.
6. The system of claim 1 wherein the plurality of predetermined detectable objects of the ordered object detection map is featureless wherein the pass determination for the item of manufacture held by the fixture mechanism is rendered from the training image data comprising featureless detectable objects.
7. The system of claim 1 , wherein the fixture mechanism comprises a conveyor assembly.
8. The system of claim 1 , wherein the fixture mechanism comprises a robot.
9. The system of claim 1 , wherein the image capture device comprises a camera.
10. The system of claim 9 , wherein the camera comprises a still image camera.
11. The system of claim 1 , wherein the fixture mechanism forms a part of a manufacturing machine, and wherein the item of manufacture comprises at least one part that is manufactured each time that the manufacturing machine is used.
12. The system of claim 11 , wherein the manufacturing machine comprises an injection molding machine.
13. The system of claim 11 , further comprising a presentation system operably coupled to the manufacturing machine that is adapted to move the at least one part from the manufacturing machine to be in operative relation to the image capture device.
14. The system of claim 13 , wherein the first control signal is sent by the programmable logic controller to the presentation system to actuate the presentation system to physically engage the at least one part for presentation to the image capture device.
15. The system of claim 1 , wherein the item of manufacture comprises a single item.
16. The system of claim 1 , wherein the image data from the visual image being identified is processed by the programable processor of the system controller to render the at least one of the pass determination and the fail determination.
17. The system of claim 1 , wherein the image data from the visual image being identified is processed by a server in communication with the system controller to render the at least one of the pass determination and the fail determination.