IP Library Granted Patent US 12,147,501
Granted Patent B2
US 12,147,501 · App. 18/533,921 · Granted Nov 19, 2024

Object detection systems and methods including an object detection model using a tailored training dataset

Inventors: Santle Camilus Kulandai Samy (Santa Clara, CA); Rajkiran Kumar Gottumukkal (Bangalore, IN); Yohai Falik (Petah Tiqwa, IL); Rajiv Ramanasankaran (Santa Clara, CA); Prantik Sen (Bangalore, IL); Deepak Chembakassery Rajendran (Kerala, IN)
Assignee: Tyco Fire & Security GmbH
G06F18/2148G06T7/246G06V10/25G06V10/62G06V10/774G06V20/41G06T2207/20081G06T2207/20084G06T2207/20132G06T2207/30168G06T2207/30196G06T2207/30232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,147,501
App. No.
18/533,921
Granted
Nov 19, 2024
Kind
B2
Abstract

Disclosed herein is an object detection system, including apparatuses and methods for object detection. An implementation may include receiving a first image frame from an ROI detection model that generated a first ROI boundary around a first object detected in the first image frame and subsequently receiving a second image frame. The implementation further includes predicting, using an ROI tracking model, that the first ROI boundary will be present in the second image frame and then detecting whether the first ROI boundary is in fact present in the second image frame. The implementation includes determining that the second image frame should be added to a training dataset for the ROI detection model when detecting that the ROI detection model did not generate the first ROI boundary in the second image frame as predicted and re-training the ROI detection model using the training dataset.

Claims (71)

1. An apparatus for object detection, comprising:

at least one memory comprising instructions;

at least one hardware processor coupled with the at least one memory and configured individually or in combination to execute the instructions, to:

detect a first object in a second image frame, wherein the first object is bounded by a first region-of-interest (ROI) boundary generated in a first image frame by an ROI detection model, and wherein the first image frame precedes the second image frame in a video;

determine that the second image frame should be added to a training dataset for the ROI detection model in response to detecting that the ROI detection model did not generate the first ROI boundary in the second image frame; and

re-train the ROI detection model, to define a re-trained ROI detection model, using the training dataset comprising the second image frame in response to determining that the second image frame should be added to the training dataset.

2. The apparatus of claim 1 , wherein the at least one hardware processor is configured to execute the instructions to:

execute the re-trained ROI detection model, wherein the re-trained ROI detection model generates the first ROI boundary around the first object in any subsequently inputted image frame depicting the first object.

3. The apparatus of claim 1 , wherein the at least one hardware processor is configured to execute the instructions to:

determine that the second image frame should not be added to the training dataset for the ROI detection model in response to detecting that the ROI detection model did generate the first ROI boundary in the second image frame; and

re-train the ROI detection model, to define the re-trained ROI detection model, using the training dataset not comprising the second image frame in response to determining that the second image frame should not be added to the training dataset.

4. The apparatus of claim 1 , wherein the at least one hardware processor is configured to execute the instructions to determine that the second image frame should be added to the training dataset by:

assigning a first tracking identifier to the first ROI boundary around the first object;

determining whether more than a threshold number of images in the training dataset include an ROI boundary assigned the first tracking identifier; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the ROI boundary assigned the first tracking identifier.

5. The apparatus of claim 1 , wherein the first object is a person and the first ROI boundary has an occluded view of the person, and wherein the at least one hardware processor is configured to execute the instructions to determine that the second image frame should be added to the training dataset by:

determining whether more than a threshold number of images in the training dataset include the occluded view of the person; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the occluded view of the person.

6. The apparatus of claim 1 , wherein the second image frame depicts a scene with a given light setting, background, or environment, and wherein the at least one hardware processor is configured to execute the instructions to determine that the second image frame should be added to the training dataset by:

determining whether more than a threshold number of images in the training dataset include the given light setting, background, or environment; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the given light setting, background, or environment.

7. The apparatus of claim 1 , wherein the at least one hardware processor is configured to execute the instructions to:

receive a third image frame from the ROI detection model, wherein the third image frame comprises a second ROI boundary around a second object, and wherein the third image frame is a subsequent frame to the second image frame;

apply a motion mask to at least the second image frame and the third image frame;

detect whether the second ROI boundary is present in the second image frame;

in response to determining that the second ROI boundary is not present in the second image frame and detecting no motion of the second object based on the motion mask, identify the second ROI boundary as a false positive;

determine that the third image frame should be added to the training dataset; and

re-train the ROI detection model using the training dataset comprising the third image frame.

8. The apparatus of claim 7 , wherein the at least one hardware processor is configured to execute the instructions to:

execute the re-trained ROI detection model, wherein the re-trained ROI detection model does not generate the second ROI boundary around the second object in any subsequently inputted image frame depicting the second object.

9. A method for object detection, comprising:

detecting a first object in a second image frame, wherein the first object is bounded by a first region-of-interest (ROI) boundary generated in a first image frame by an ROI detection model, and wherein the first image frame precedes the second image frame in a video;

determining that the second image frame should be added to a training dataset for the ROI detection model in response to detecting that the ROI detection model did not generate the first ROI boundary in the second image frame; and

re-training the ROI detection model, to define a re-trained ROI detection model, using the training dataset comprising the second image frame in response to determining that the second image frame should be added to the training dataset.

10. The method of claim 9 , further comprising:

executing the re-trained ROI detection model, wherein the re-trained ROI detection model generates the first ROI boundary around the first object in any subsequently inputted image frame depicting the first object.

11. The method of claim 9 , further comprising:

determining that the second image frame should not be added to the training dataset for the ROI detection model in response to detecting that the ROI detection model did generate the first ROI boundary in the second image frame; and

re-training the ROI detection model, to define the re-trained ROI detection model, using the training dataset not comprising the second image frame in response to determining that the second image frame should not be added to the training dataset.

12. The method of claim 9 , wherein determining that the second image frame should be added to the training dataset comprises:

assigning a first tracking identifier to the first ROI boundary around the first object;

determining whether more than a threshold number of images in the training dataset include an ROI boundary assigned the first tracking identifier; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the ROI boundary assigned the first tracking identifier.

13. The method of claim 9 , wherein the first object is a person and the first ROI boundary has an occluded view of the person, and wherein determining that the second image frame should be added to the training dataset comprises:

determining whether more than a threshold number of images in the training dataset include the occluded view of the person; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the occluded view of the person.

14. The method of claim 9 , wherein the second image frame depicts a scene with a given light setting, background, or environment, and wherein determining that the second image frame should be added to the training dataset comprises:

determining whether more than a threshold number of images in the training dataset include the given light setting, background, or environment; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the given light setting, background, or environment.

15. The method of claim 9 , further comprising:

receiving a third image frame from the ROI detection model, wherein the third image frame comprises a second ROI boundary around a second object, and wherein the third image frame is a subsequent frame to the second image frame;

applying a motion mask to at least the second image frame and the third image frame;

detecting whether the second ROI boundary is present in the second image frame;

in response to determining that the second ROI boundary is not present in the second image frame and detecting no motion of the second object based on the motion mask, identifying the second ROI boundary as a false positive;

determining that the third image frame should be added to the training dataset; and

re-training the ROI detection model using the training dataset comprising the third image frame.

16. The method of claim 15 , further comprising:

executing the re-trained ROI detection model, wherein the re-trained ROI detection model does not generate the second ROI boundary around the second object in any subsequently inputted image frame depicting the second object.

17. A non-transitory computer readable medium storing thereon computer executable instructions for object detection, including instructions for:

detecting a first object in a second image frame, wherein the first object is bounded by a first region-of-interest (ROI) boundary generated in a first image frame by an ROI detection model, and wherein the first image frame precedes the second image frame in a video;

determining that the second image frame should be added to a training dataset for the ROI detection model in response to detecting that the ROI detection model did not generate the first ROI boundary in the second image frame; and

re-training the ROI detection model, to define a re-trained ROI detection model, using the training dataset comprising the second image frame in response to determining that the second image frame should be added to the training dataset.

18. The non-transitory computer readable medium of claim 17 , further comprising instructions for:

executing the re-trained ROI detection model, wherein the re-trained ROI detection model generates the first ROI boundary around the first object in any subsequently inputted image frame depicting the first object.

19. The non-transitory computer readable medium of claim 17 , further comprising instructions for:

determining that the second image frame should not be added to the training dataset for the ROI detection model in response to detecting that the ROI detection model did generate the first ROI boundary in the second image frame; and

re-training the ROI detection model, to define the re-trained ROI detection model, using the training dataset not comprising the second image frame in response to determining that the second image frame should not be added to the training dataset.

20. The non-transitory computer readable medium of claim 17 , wherein an instruction for determining that the second image frame should be added to the training dataset comprises instructions for:

assigning a first tracking identifier to the first ROI boundary around the first object;

determining whether more than a threshold number of images in the training dataset include an ROI boundary assigned the first tracking identifier; and

adding the second image frame to the training dataset in response to determining that less than the threshold number of images in the training dataset include the ROI boundary assigned the first tracking identifier.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE 3RD INVENTORS NAME PREVIOUSLY RECORDED AT REEL: 65879 FRAME: 792. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNORS INTEREST. Recorded Sep 13, 2024
From: KULANDAISAMY, SANTLE CAMILUS; GOTTUMUKKAL, RAJKIRAN KUMAR; FALIK, YOHAY; RAMANASANKARAN, RAJIV; RAJENDRAN, DEEPAK CHEMBAKASSERY; SEN, PRANTIK
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 068967/0401 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2024
From: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
To: TYCO FIRE & SECURITY GMBH
Reel/Frame 068494/0384 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2023
From: KULANDAI SAMY, SANTLE CAMILUS; GOTTUMUKKAL, RAJKIRAN KUMAR; FALIK, YOHAI; RAMANASANKARAN, RAJIV; RAJENDRAN, DEEPAK CHEMBAKASSERY; SEN, PRANTIK
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 065879/0792 →