System and method for deep network fusion for fast and robust object detection

1. A method, comprising:

receiving an image;

detecting an object in the image;

determining, by a primary object detector, a primary confidence detection score of the object;

determining, by a classification network, a confidence scaling factor of the object;

adjusting the primary confidence detection score based on multiplying the primary confidence detection score by the confidence scaling factor;

generating a semantic segmentation mask based on classifying each pixel into one of pre-determined classes;

determining an amount of overlap, if any, between a bounding box of the object and the semantic segmentation mask; and

adjusting the primary confidence detection score and eliminating a secondary object detection from the semantic segmentation mask if there is no overlap between the bounding box of the object and the semantic segmentation mask,

wherein the semantic segmentation mask is a binary mask generated with one color showing activated pixels in the image and another color showing a background in the image.

2. The method of claim 1 , further comprising:

scaling the primary confidence detection score based on multiplying the primary confidence detection score by a product of a plurality of confidence scaling factors of the object from a plurality of classification networks.

3. The method of claim 2 , further comprising:

setting an object detection to a fixed size using the scaled primary confidence detection score;

determining random crops with the fixed size from the scaled primary confidence detection score; and

labelling a class of the random crops with respect to ground truth to train the classification network.

4. The method of claim 1 , wherein the confidence scaling factor is based on a ratio of a classification probability of the object to a desired maximum classification probability.

5. The method of claim 1 , wherein the confidence scaling factor is set to at least a desired minimum value.

6. The method of claim 1 , further comprising determining a secondary confidence detection score of the object;

maintaining the primary confidence detection score if the secondary confidence detection score is greater than or equal to a threshold; and

adjusting the primary confidence detection score if the secondary confidence detection score is less than the threshold.

7. The method of claim 6 , wherein adjusting the primary confidence detection score if the secondary confidence detection score is less than the threshold comprises multiplying the primary confidence detection score by a product of the secondary confidence detection score and a plurality of confidence scaling factors.

8. The method of claim 6 , wherein adjusting the primary confidence detection score if the secondary confidence detection score is less than the threshold comprises multiplying the adjusted primary confidence detection score by a product of the secondary confidence detection score and a plurality of confidence scaling factors.

9. The method of claim 1 , further comprising adjusting the primary confidence detection score in proportion to an amount of overlap if the amount of overlap between a bounding box of the object and a semantic segmentation mask is less than a threshold.

10. The method of claim 1 , further comprising maintaining the primary confidence detection score if an amount of overlap between a bounding box of the object and a semantic segmentation mask is greater than or equal to a threshold.

11. The method of claim 1 , further comprising operating the primary object detector and the classification network in parallel.

12. An apparatus, comprising:

an image sensor;

a primary object detector;

a classification network; and

a processor configured to:

capture an image from the image sensor,

detect an object in the image,

determine a primary confidence detection score of the object using the primary object detector,

determine a confidence scaling factor of the object using the classification network,

adjust the primary confidence detection score based on multiplying the primary confidence detection score by the confidence scaling factor,

generate a semantic segmentation mask based on classifying each pixel into one of pre-determined classes,

determine an amount of overlap, if any, between a bounding box of the object and the semantic segmentation mask; and

adjust the primary confidence detection score and eliminate a secondary object detection from the semantic segmentation mask if there is no overlap between the bounding box of the object and the semantic segmentation mask,

wherein the semantic segmentation mask is a binary mask generated with one color showing activated pixels in the image and another color showing a background in the image.

13. The apparatus of claim 12 , wherein the processor is further configured to scale the primary confidence detection score based on multiplying the primary confidence detection score by a product of a plurality of confidence scaling factors of the object from a plurality of classification networks.

14. The apparatus of claim 12 , wherein the confidence scaling factor is based on a ratio of a classification probability of the object to a desired maximum classification probability.

15. The apparatus of claim 12 , wherein the confidence scaling factor is set to at least a desired minimum value.

16. The apparatus of claim 12 , wherein the processor is further configured to determine a secondary confidence detection score of the object,

maintain the primary confidence detection score if the secondary confidence detection score is greater than or equal to a threshold, and

adjust the primary confidence detection score if the secondary confidence detection score is less than the threshold.

17. The apparatus of claim 16 , wherein adjusting the primary confidence detection score if the secondary confidence detection score is less than the threshold comprises multiplying the primary confidence detection score by a product of the secondary confidence detection score and a plurality of confidence scaling factors.

18. The apparatus of claim 16 , wherein adjusting the primary confidence detection score if the secondary confidence detection score is less than the threshold comprises multiplying the adjusted primary confidence detection score by a product of the secondary confidence detection score and a plurality of confidence scaling factors.

19. The apparatus of claim 12 , wherein the processor is further configured to adjust the primary confidence detection score in proportion to an amount of overlap if the amount of overlap between a bounding box of the object and a semantic segmentation mask is less than a threshold.

20. The apparatus of claim 12 , wherein the processor is further configured to maintain the primary confidence detection score if an amount of overlap between a bounding box of the object and a semantic segmentation mask is greater than or equal to a threshold.

21. The apparatus of claim 12 , wherein the processor is further configured to operate the primary object detector and the classification network in parallel.

22. The apparatus of claim 12 , wherein the processor is further configured to:

set an object detection to a fixed size using the scaled primary confidence detection score,

determine random crops with the fixed size from the scaled primary confidence detection score, and

label a class of the random crops with respect to ground truth to train the classification network.

23. A method of manufacturing a processor, comprising:

forming the processor as part of a wafer or package that includes at least one other processor, wherein the processor is configured to capture an image from an image sensor, detect an object in the image, determine a primary confidence detection score of the object using a primary object detector, determine a confidence scaling factor of the object using a classification network, adjust the primary confidence detection score based on multiplying the primary confidence detection score by the confidence scaling factor, generate a semantic segmentation mask based on classifying each pixel into one of pre-determined classes, determine an amount of overlap, if any, between a bounding box of the object and the semantic segmentation mask, and adjust the primary confidence detection score and eliminate a secondary object detection from the semantic segmentation mask if there is no overlap between the bounding box of the object and the semantic segmentation mask, wherein the semantic segmentation mask is a binary mask generated with one color showing activated pixels in the image and another color showing a background in the image.

24. A method of constructing an integrated circuit, comprising:

generating a mask layout for a set of features for a layer of the integrated circuit, wherein the mask layout includes standard cell library macros for one or more circuit features that include a processor configured to capture an image from a image sensor, detect an object in the image, determine a primary confidence detection score of the object using a primary object detector, determine a confidence scaling factor of the object using a classification network, adjust the primary confidence detection score based on multiplying the primary confidence detection score by the confidence scaling factor, generate a semantic segmentation mask based on classifying each pixel into one of pre-determined classes, determine an amount of overlap, if any, between a bounding box of the object and the semantic segmentation mask, and adjust the primary confidence detection score and eliminate a secondary object detection from the semantic segmentation mask if there is no overlap between the bounding box of the object and the semantic segmentation mask, wherein the semantic segmentation mask is a binary mask generated with one color showing activated pixels in the image and another color showing a background in the image.