IP Library › Granted Patent US 12,008,816
Granted Patent B2
US 12,008,816 · App. 17/540,172 · Granted Jun 11, 2024

Method and system for real time object detection

Inventors: Vipin Sharma (Haryana, IN); Akshat Agrawal (Haryana, IN); Jitesh Kumar Singh (Karnataka, IN); Arpit Awasthi (Haryana, IN)
Assignee: HL KLEMOVE CORP.
G06V20/56G06F18/24G06N3/04G06V10/32G06V10/40G06V10/95
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,008,816
App. No.
17/540,172
Granted
Jun 11, 2024
Kind
B2
Abstract

The present disclosure relates to a method for real-time object detection, the method comprising: capturing an image in vicinity of a vehicle; feeding the captured image to a deep fully convolution neural network; extracting one or more relevant features from the captured image; classifying the extracted features using one or more branches to identify different size of objects; predicting objects present in the image based on a predetermined confidence threshold; and marking the predicted objects in the image.

Claims (39)

1. A method for real-time object detection for a host vehicle, the method comprising:

capturing an image in vicinity of the host vehicle;

feeding the captured image to a deep fully convolution neural network;

extracting one or more relevant features from the captured image;

classifying the extracted features using one or more branches of the deep fully convolution neural network to identify different size of objects, each of the one or more branches comprising a different receptive field corresponding to the size of the object;

predicting objects present in the image based on a predetermined confidence threshold;

marking the predicted objects in the image; and

plotting the marked image on a display,

wherein the different receptive field is created by performing a different number of down sampling and a different number of depth-wise separable convolution to the extracted features.

2. The method of claim 1 , wherein the captured image is a Ground Truth (GT) image marked using a Bounding Box annotation tool.

3. The method of claim 1 , further comprising reshaping the captured images into a predetermined compatible size, while still maintaining the aspect ratio of the objects present in the image which in turn is fed to the deep fully convolution neural network.

4. The method of claim 1 , wherein classifying includes routing the object having a smaller size early off for the prediction in the deep fully convolution neural network.

5. The method of claim 1 , wherein the down sampling comprises:

down sampling a feature map, wherein the feature map is extracted from an image from a plurality of feature vectors, to record the most activated features of the image; and

simultaneously down sampling the same feature map to maintain a localization information,

wherein the deep fully convolution neural network comprises down sampling-convolution-receptive block (DCR) technique.

6. The method of claim 1 , wherein the predicting objects comprises:

comparing a confidence score associated with each of intermediary prediction objects to the predetermined confidence threshold; and

choosing predicted objects having the score above or equal to the predetermined confidence threshold.

7. A system for real-time object detection for a host vehicle, comprising:

an image sensor configured to capture an image in the vicinity of the host vehicle;

a controller communicatively connected to the image sensor and configured to:

obtain the captured from the image sensor;

feed the captured image to a deep fully convolution neural network;

extract one or more relevant features from the captured image;

classify the extracted features using one or more branches of the deep fully convolution neural network to identify different size of objects, each of the or more branches comprising a different receptive field corresponding to the size of the object;

predict objects present in the image based on a predetermined confidence threshold;

mark the predicted objects in the image; and

plot the marked image on a display,

wherein the different receptive field is created by performing a different number of down sampling and a different number of depth-wise separable convolution to the extracted features.

8. The system of claim 7 , wherein the controller is further configured to reshape the captured images into a predetermined compatible size while still maintaining the aspect ratio of the objects present in the image to be fed to the deep fully convolution neural network.

9. The system of claim 7 , wherein the controller is further configured to route the object having a smaller size early off, for the prediction in the deep fully convolution neural network.

10. The system of claim 7 , wherein the down sampling comprises:

down sampling a feature map, wherein the feature map is extracted from an image from a plurality of feature vectors, to record the most activated features of the image; and

simultaneously down sampling the same feature map to maintain a localization information,

wherein the deep fully convolution neural network comprises down sampling-convolution-receptive block (DCR) technique.

11. The system for object detection of claim 7 , wherein the controller is further configured to:

compare a confidence score associated with each of intermediary prediction objects to the predetermined confidence threshold; and

choose predicted objects having the score above or equal to the predetermined confidence threshold.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2022
From: MANDO CORPORATION
To: HL KLEMOVE CORP.
Reel/Frame 060866/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2021
From: SHARMA, VIPIN; AGRAWAL, AKSHAT; SINGH, JITESH KUMAR; AWASTHI, ARPIT
To: MANDO CORPORATION
Reel/Frame 058294/0781 →
Priority Claims (1)
IN 202011052691 · Dec 3, 2020 · national
Continuity (1)
Related Publication 20220180107A1 · Jun 9, 2022