IP Library Granted Patent US 10,229,505
Granted Patent B2
US 10,229,505 · App. 15/205,396 · Granted Mar 12, 2019

Motion determination system and method thereof

Inventors: Dajun Ding (Shanghai, CN); Lili Zhao (Shanghai, CN); Taiqiang Cao (Shanghai, CN)
Assignee: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
G06T7/246G06K9/00791G06T7/20G06T7/215G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,229,505
App. No.
15/205,396
Granted
Mar 12, 2019
Kind
B2
Abstract

Methods and apparatus for determining the motion of a movable object are disclosed. The methods may include, in a view of the environment outside the movable object, identifying a first region of interest (ROI) corresponding to a static portion of the environment. The methods may also include, in the view of the environment, identifying a second region of interest (ROI) corresponding to an active portion of the environment. The methods may also include receiving first and second image data respectively representing the first and second ROIs. The methods may also include analyzing the first image data over time. The methods may also include analyzing the second image data over time. The methods may further include determining whether the movable object is in motion based on the analyzes of the first and second image data.

Claims (73)

1. A method performed by a processor in communication with an imaging device mounted on a movable object, the method comprising:

in a view of an environment outside the movable object, identifying a first region of interest (ROI) corresponding to a static portion of the environment, the first ROI covering at least an area in which vehicle and pedestrian traffic cannot be observed;

in the view of the environment, identifying a second region of interest (ROI) corresponding to an active portion of the environment, the second ROI covering at least an area in which vehicle or pedestrian traffic can be observed;

receiving, from the imaging device, first and second image data respectively representing the first and second ROIs;

analyzing the first image data over time, wherein analyzing the first image data comprises performing a first comparison of an image property of the first ROI with a first threshold;

analyzing the second image data over time, wherein analyzing the second image data comprises performing a second comparison of an image property of the second ROI with a second threshold; and

determining whether the movable object is in motion based on the analyses of the first and second image data.

2. The method of claim 1 , wherein the determining comprises concluding that the movable object is not in motion if a change over time of the first image data is less than a change over time of the second image data.

3. The method of claim 1 , wherein

the movable object is a vehicle.

4. The method of claim 1 , wherein the first ROI corresponds to a first static portion of the environment, the method further comprising:

in the view of the environment, identifying a third ROI corresponding to a second static portion of the environment, second static portion being non-overlapping with the first static portion;

receiving third image data representing the third ROI;

analyzing the third image data over time;

comparing the analyses of the first and third image data; and

determining whether the movable object is in motion at least partially based on the comparison result.

5. The method of claim 1 , wherein:

determining whether the movable object is in motion comprises determining whether the movable object is in motion based on at least one of the first or second comparisons.

6. The method of claim 5 , wherein the image property of a ROI comprises at least one of a change of pixel values over time, a complexity of pixel values in space, or a brightness in the ROI.

7. The method of claim 6 , wherein the change of pixel values over time is calculated as a change over time of an averaged pixel value.

8. The method of claim 6 , wherein the complexity of pixel values is calculated as a root-mean-square derivation of pixel values.

9. The method of claim 6 , wherein the brightness in the ROI is calculated as an averaged pixel brightness value.

10. The method of claim 1 , wherein receiving the first and second image data comprises:

receiving raw data corresponding to the first and second ROIs; and

down-sampling the raw data to generate the first and second image data.

11. The method of claim 10 , wherein:

the raw data corresponding to the first and second ROIs includes a plurality of pixel values; and

down-sampling the raw data comprises:

sampling, according to a sampling rule, a subset of pixel values from the raw data corresponding to the first and second ROIs;

dividing the sampled pixel values into a predetermined number of groups; and

averaging the pixel values in the groups.

12. The method of claim 6 , further comprising:

determining whether a change of pixel values over time in the first ROI exceeds a first amount of change; and

when the change of pixel values over time in the first ROI exceeds the first amount of change, concluding that the movable object is in motion.

13. The method of claim 12 , further comprising:

when the change of pixel values over time in the first ROI does not exceed the first amount of change, determining whether a complexity of pixel values in the first ROI exceeds a first complexity level; and

when the complexity of pixel values in the first ROI exceeds the first complexity level, concluding that the movable object is not in motion.

14. The method of claim 13 , further comprising:

when the complexity of pixel values in the first ROI does not exceed the first complexity level, determining whether a brightness in the first ROI exceeds a first brightness level;

when the brightness in the first ROI does not exceed the first brightness level, determining whether a brightness in the second ROI exceeds a second brightness level; and

when the brightness of the second ROI exceeds the second brightness level, determining that the movable object is in motion.

15. The method of claim 14 , further comprising:

when the brightness in the second ROI exceeds the second brightness level, concluding that the current time is nighttime.

16. The method of claim 14 , further comprising:

when the brightness in the second ROI does not exceed the second brightness level, concluding that the movable object is not in motion.

17. The method of claim 14 , further comprising:

when the brightness in the second ROI does not exceed the second brightness level, concluding that the movable object is in at least one of nighttime or a covered space.

18. The method of claim 14 , further comprising:

when the brightness in the first ROI exceeds the first brightness level, concluding that the movable object is in motion.

19. The method of claim 14 , further comprising:

when the brightness in the first ROI exceeds the first brightness level, concluding that the movable object is in daytime.

20. The method of claim 1 , further comprising:

determining a distance between the movable object and a second object outside the movable object;

analyzing the distance over time; and

determining a current state of the movable object based on the analysis of the distance.

21. The method of claim 20 , further comprising:

when the distance stays below a first threshold distance for longer than a first time duration, concluding that the movable object is not in motion.

22. A device, comprising:

a memory storing instructions; and

a processor in communication with an imaging device mounted on a movable object, the processor configured to execute the instructions to:

in a view of an environment outside the movable object, identify a first region of interest (ROI) corresponding to a static portion of the environment, the first ROI covering at least an area in which vehicle and pedestrian traffic cannot be observed;

in the view of the environment, identify a second region of interest (ROI) corresponding to an active portion of the environment, the second ROI covering at least an area in which vehicle or pedestrian traffic can be observed;

receive, from the imaging device, first and second image data respectively representing the first and second ROIs;

analyze the first image data over time, wherein analyzing the first image data comprises performing a first comparison of an image property of the first ROI with a first threshold;

analyze the second image data over time, wherein analyzing the second image data comprises performing a second comparison of an image property of the second ROI with a second threshold; and

determine whether the movable object is in motion based on the analyses of the first and second image data.

23. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor in communication with an imaging device mounted on a movable object, cause the processor to perform operations comprising:

in a view of an environment outside the movable object, identifying a first region of interest (ROI) corresponding to a static portion of the environment, the first ROI covering at least an area in which vehicle and pedestrian traffic cannot be observed;

in the view of the environment, identifying a second region of interest (ROI) corresponding to an active portion of the environment, the second ROI covering at least an area in which vehicle or pedestrian traffic can be observed;

receiving, from the imaging device, first and second image data respectively representing the first and second ROIs;

analyzing the first image data over time, wherein analyzing the first image data comprises performing a first comparison of an image property of the first ROI with a first threshold;

analyzing the second image data over time, wherein analyzing the second image data comprises performing a second comparison of an image property of the second ROI with a second threshold; and

determining whether the movable object is in motion based on the analyses of the first and second image data.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Nov 5, 2024
From: EAST WEST BANK
To: KAMI VISION INCORPORATED
Reel/Frame 070792/0551 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 28, 2022
From: KAMI VISION INCORPORATED
To: EAST WEST BANK
Reel/Frame 059512/0101 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
To: KAMI VISION INC.
Reel/Frame 059275/0652 →
CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF THE THIRD ASSIGNOR TO TAIQIANG CAO PREVIOUSLY RECORDED ON REEL 047674 FRAME 0011. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 16, 2019
From: DING, DAJUN; ZHAO, LILI; CAO, TAIQIANG
To: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 048080/0899 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S NAME TO "SHANGHAI XIAOYI TECHNOLOGY CO., LTD." PREVIOUSLY RECORDED ON REEL 039110 FRAME 0072. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 19, 2018
From: DING, DAJUN; ZHAO, LILI; TAIQIANG, CAO
To: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 047674/0011 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2016
From: DING, DAJUN; ZHAO, LILI; CAO, TAIQIANG
To: XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 039110/0072 →
Priority Claims (1)
CN 2016 1 0496592 · Jun 29, 2016 · national
Continuity (1)
Related Publication 20180005384A1 · Jan 4, 2018
Cited By (2)
US 12,361,721 US 12,499,555