IP Library Granted Patent US 10,346,685
Granted Patent B2
US 10,346,685 · App. 15/212,761 · Granted Jul 9, 2019

System and method for detecting and tracking a moving object

Inventors: Dajun Ding (Shanghai, CN); Lili Zhao (Shanghai, CN)
Assignee: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
G06K9/00624G06T7/254G06T2207/10004G06T2207/20021G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,346,685
App. No.
15/212,761
Granted
Jul 9, 2019
Kind
B2
Abstract

A device includes a memory configured to store instructions and a processor configured to execute the instructions to obtain image data of a region of interest included in an image frame. The processor may also be configured to compare the image data of the region of interest with image data of a background to detect a change in the region of interest. The processor may further be configured to detect the object in image frame based on the detected change.

Claims (64)

1. A device, comprising:

a memory configured to store instructions; and

a processor configured to execute the instructions to:

obtain image data of a plurality of predetermined regions of interest within a plurality of image frames, wherein at least one of the regions of interest is arranged at the same position in every one of the plurality of image frames;

compare the image data of the regions of interest with image data of backgrounds to detect changes in the regions of interest;

obtain time data indicating when the detected changes occurred in the regions of interest;

compare the time data of the detected changes to determine a time sequence of the detected changes in the regions of interest; and

determine a movement characteristic of an object in the plurality of image frames based on the determine time sequence of the detected changes in the regions of interest.

2. The device of claim 1 , wherein the processor is further configured to execute the instructions to detect the object in the plurality of image frames based on:

the time data indicating when the detected changes occurred in the regions of interest, and

at least one of brightness data or color data obtained from locations within the regions of interest where the changes occurred.

3. The device of claim 1 , wherein the processor is further configured to execute the instructions to:

determine, based on the comparison of the image data of the regions of interest with the image data of the backgrounds, a difference between the image data of at least one of the regions of interest and the image data of at least one of the backgrounds.

4. The device of claim 3 , wherein the processor is further configured to execute the instructions to:

determine a difference vector between a first vector storing the image data of the at least one region of interest and a second vector storing the image data of the at least on background; and

determine whether the difference vector satisfies a predetermined condition.

5. The device of claim 3 , wherein the processor is further configured to execute the instructions to:

determine that the difference is smaller than a predetermined difference value; and

update the image data of the at least one background using the image data of the at least one region of interest.

6. The device of claim 5 , wherein updating the image data of the at least one background includes:

updating the image data of the at least one background using image data of the at least one region of interest from a predetermined number of image frames.

7. The device of claim 3 , wherein the processor is further configured to execute the instructions to:

determine that the difference is greater than or equal to a predetermined difference value;

determine a number of image frames, from a predetermined plurality of image frames, in which the difference is greater than or equal to the predetermined difference value; and

when the number of image frames is greater than or equal to a predetermined frame number value, update the image data of the at least one background using the image data of the at least one region of interest from the predetermined plurality of image frames.

8. The device of claim 1 , wherein the processor is further configured to execute the instructions to:

determine, from the comparison of the image data of the regions of interest with the image data of the backgrounds, a location in each of the plurality of regions of interest where a change has occurred;

obtain color data at the location in different regions of interests from a different image frames; and

compare color data obtained from the different regions of interest in the different image frames.

9. The device of claim 1 , wherein the processor is further configured to execute the instructions to:

determine a moving direction and a moving speed of the object across the regions of interest based on the time sequence of the detected changes in the regions of interest.

10. The device of claim 9 , wherein the processor is further configured to execute the instructions to:

calculate a plurality of time intervals based on a plurality of pairs of adjacent time instances;

compare the time intervals with a plurality of predetermined time delay values; and

determine that a same object is moving across the regions of interest in the moving direction when:

each of the time intervals is less than each of the corresponding predetermined time delay values, and

a difference in the color data obtained from the different regions of interest in the different image frames is smaller than a predetermined color value.

11. The device of claim 10 , wherein the processor is further configured to execute the instructions to initiate tracking of the object in the moving direction by sending a control signal to drive a motor to turn a camera in the moving direction to follow the object.

12. The device of claim 11 , wherein the processor is further configured to execute the instructions to determine the moving speed of the object based on the time instances and distances between adjacent regions of interest, and wherein the control signal controls the speed of driving the motor based on the moving speed.

13. The device of claim 1 , wherein the processor is further configured to execute the instructions to:

obtain a brightness matrix including brightness values of pixels included in at least one of the regions of interest; and

transform the brightness matrix into a brightness vector, each row of the vector being calculated based on brightness values of all columns in the row,

wherein the image data of the at least one region of interest is represented by the brightness vector.

14. The device of claim 13 , wherein the processor is further configured to execute the instructions to:

obtain a color data matrix including color data of pixels included in the at least one region of interest; and

transform the color data matrix into a color value vector, each row of the vector being a sum of color values of all columns in the row,

wherein the image data associated with the at least one region of interest includes the color value vector.

15. The device of claim 13 , wherein the processor is further configured to execute the instructions to:

obtain an initial brightness vector as the image data of at least one of the backgrounds, based on brightness vectors obtained from a predetermined number of image frames.

16. The device of claim 1 , wherein the plurality of image frames comprises three regions of interest.

17. A method for tracking an object, comprising:

obtaining image data of a plurality of predetermined regions of interest within a plurality of image frames, wherein at least one of the regions of interest is arranged at the same position in every one of the plurality of image frames;

comparing the image data of the regions of interest with image data of backgrounds to detect changes in the regions of interest;

obtaining time data indicating when the detected changes occurred in the regions of interest;

comparing the time data of the detected changes to determine a time sequence of the detected changes in the regions of interest and

tracking movement of an object in the plurality of image frames based on the determined time sequence of the detected changes in the regions of interest.

18. The method of claim 17 , wherein tracking movement of the object in the plurality of image frames includes detecting the object based on: the time data indicating when the detected changes occurred in the regions of interest, and color data obtained from a location within the plurality of regions of interest.

19. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method for tracking an object, the method comprising:

obtaining image data of a plurality of predetermined regions of interest within a plurality of image frames, wherein at least one of the regions of interest is arranged at the same position in every one of the plurality of image frames;

comparing the image data of the regions of interest with image data of backgrounds to detect changes in the regions of interest;

obtaining time data indicating when the detected changes occurred in the regions of interest;

comparing the time data of the detected changes to determine a time sequence of the detected changes in the regions of interest; and

tracking movement of an object in the plurality of image frames based on the determine time sequence of the detected changes in the regions of interest.

20. The non-transitory computer-readable storage medium of claim 19 , wherein tracking movement of the object in the plurality of image frames includes detecting the object based on: the time data indicating when the detected changes occurred in the regions of interest, and color data obtained from a location within the plurality of regions of interest.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Nov 5, 2024
From: EAST WEST BANK
To: KAMI VISION INCORPORATED
Reel/Frame 070792/0551 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 28, 2022
From: KAMI VISION INCORPORATED
To: EAST WEST BANK
Reel/Frame 059512/0101 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
To: KAMI VISION INC.
Reel/Frame 059275/0652 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S NAME TO "SHANGHAI XIAOYI TECHNOLOGY CO., LTD" . PREVIOUSLY RECORDED ON REEL 039193 FRAME 0261. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNORS HEREBY CONFIRMS THE ASSIGNMENT. Recorded May 31, 2018
From: DING, DAJUN; ZHAO, LILI
To: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 047169/0402 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2016
From: DING, DAJUN; ZHAO, LILI
To: XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 039193/0261 →
Priority Claims (1)
CN 2016 1 0498023 · Jun 29, 2016 · national
Continuity (1)
Related Publication 20180005033A1 · Jan 4, 2018
Cited By (4)
US 12,260,023 US 12,306,301 US 12,607,515 US 12,693,717