Vision system for object detection, recognition, classification and tracking and the method thereof
Aspects of the present disclosure are directed to, for example, a method for object detection, recognition, classification and tracking using a distributed networked architecture. In some embodiments, the distributed network architecture may include one or more sensor units wherein the image acquisition and the initial feature extraction are performed and a gateway processor for further data processing. Some aspects of the present disclosure are also directed to a vision system for object detection, and to algorithms implemented in the vision system for executing the method acts for object detection, recognition, classification and/or tracking.
1. A method for object detection comprising:
acts performed by a sensor unit including
acquiring an image from a camera,
performing image pre-processing on the acquired image to generate a pre-processed image,
performing detection and identification of objects in the pre-processed image using a computer vision detection algorithm,
performing data feature extraction on the detected and identified object(s) in the pre-processed image using a computer vision data feature extraction algorithm (DFE algorithm) to generate a reduced dataset comprising extracted data features,
transmitting the reduced dataset to a gateway processor, and
acts performed by a gateway processor including
receiving a reduced dataset on the gateway processor.
2. The method of claim 1 , wherein the act of performing objection detection is performed using a single image.
3. The method of claim 1 , wherein the act of performing image pre-processing on the acquired image includes
obtaining one or more sub-frame images within a full-frame image where the full-frame image is the acquired image, and
generating pre-processed image(s) of the one or more sub-frame images.
4. The method of claim 1 further including, using the gateway processor, to perform object recognition and/or object classification by feeding the reduced dataset into a machine learning model and executing a machine learning algorithm adapted to perform object recognition and/or object classification based on the reduced dataset.
5. The method of claim 1 , further including acts performed in the sensor unit and/or in the gateway processor of:
acquiring a pixel object height of a detected object, and
comparing the pixel object height with tabulated physical object height(s) and tabulated camera parameter(s),
to approximate a distance of the detected object(s) to the camera.
6. The method of claim 5 , further including the step of, in the gateway processor or in the sensor unit, acquiring a feature point and a centre point in a feature plane, the feature plane being parallel to the image plane of the sensor unit, to approximate an object-camera angle from the centre point to the feature point.
7. The method of claim 6 , wherein the approximate object-camera angle and the approximate object-camera distance are combined to approximate a localization of object from a single camera.
8. The method of claim 7 , wherein a distance between localized objects is detected.
9. The method of claim 7 , wherein at least two sensor units are used and wherein the approximate object-camera distance and/or approximate object-camera angle from respective sensor units are used to self-calibrate a localisation of the object.
10. The method of claim 9 , further including the step of using the gateway processor to coordinate which of the at least two sensor units are used to self-calibrate the localisation of the object.
11. A sensor unit comprising:
a camera configured and arranged to acquire an image,
pre-processor means configured and arranged to
perform image pre-processing on the acquired image to generate a pre-processed image,
perform detection and identification of objects jn the pre-processed image, and
perform data feature extraction on the detected and identified objects in the pre-processed image for generating a reduced dataset comprising extracted data features, and
sensor communication means configured and arranged for transmitting the reduced dataset from the pre-processor means.
12. The sensor unit of claim 11 , wherein the pre-processor means are configured and arranged to perform object detection in a single image.
13. A vision system for object detection comprising:
a gateway processor connected to a computer-readable medium
the computer-readable medium including one or more computer program products, and
one or more sensor units, each sensor unit connected to the computer-readable medium and including sensor communication means configured and arranged for transmitting a reduced dataset to the gateway processor,
wherein the vision system is configured and arranged to
acquire an image from a camera,
perform image pre-processing on the acquired image to generate a pre-processed image,
perform detection and identification of objects in the pre-processed image using a computer vision detection algorithm, and
perform data feature extraction on the detected and identified object(s) in the pre-processed image using a computer vision data feature extraction algorithm (DFE algorithm) to generate the reduced dataset comprising extracted data features.
14. The vision system according to claim 13 , wherein the gateway processor is configured and arranged to perform object recognition and/or object classification by feeding the reduced dataset into a machine learning model executing a machine learning algorithm adapted to perform object recognition and/or object classification based on the reduced dataset.
15. The vision system according to claim 14 , wherein the gateway processor is configured and arranged to perform the acts of:
acquiring a pixel object height of a detected object, and
comparing the pixel object height with tabulated physical object height(s) and tabulated camera parameter(s),
to approximate a distance of the detected object(s) to the camera.
16. The vision system according to claim 15 , wherein the gateway processor is configured and arranged to acquire a feature point and a centre point in a feature plane, the feature plane being parallel to the image plane of the sensor unit, to approximate an object-camera angle from the centre point to the feature point.
17. The vision system according to claim 13 , wherein the one or more sensor units includes at least two sensor units configured and arranged to approximate object-camera distance and/or approximate object-camera angle from respective ones of the at least two one or more sensor units and used to self-calibrate for localisation of the object.
18. The vision system according to claim 17 , wherein the gateway processor is configured and ranged to coordinate which of the at least two sensor units are used to self-calibrate for the localisation of the object.
19. The vision system according to claim 13 , the vision system is configured and arranged to be operated in two or more states.
20. The vision system according to claim 13 , wherein the gateway processor is within a sensor unit.
21. The vision system for object detection according to claim 13 , further including at least two sensor units wherein a first sensor unit is operated with a field of view being separate from a field of view of a second sensor unit.