IP Library › Granted Patent US 10,616,486
Granted Patent B2
US 10,616,486 · App. 16/059,993 · Granted Apr 7, 2020

Video stabilization

Inventors: Enrique Corona (Canton, MI); Stephen Giardinelli (Warren, MI); Bruno Sielly Jales Costa (Santa Clara, CA); Mostafa Parchami (Dearborn, MI); Gintaras Vincent Puskorius (Novi, MI)
Assignee: FORD GLOBAL TECHNOLOGIES, LLC
H04N5/23267G06T7/20G06T7/73H04N5/23258
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,616,486
App. No.
16/059,993
Granted
Apr 7, 2020
Kind
B2
Abstract

A computing system can receive a stabilized image stream, wherein the stabilized image stream is drift-corrected based on determining that an input image stream is stable and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to the real world. The computing system can operate a vehicle based on determining at least one moving object in the stabilized image stream.

Claims (26)

1. A method, comprising:

receiving a video data stream, wherein the video data stream is drift-corrected based on determining that an input video data stream is stable based on determining an eccentricity ε k at a time k and comparing the eccentricity ε k to an empirically determined threshold value, where eccentricity ε k measures a rate at which data points associated with a pixel location x k are changing as a function of time k, and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to real world coordinates; and

operating a vehicle based on determining at least one moving object in a stabilized image stream.

2. The method of claim 1 , wherein the eccentricity ε k is based on recursively processing the input video data stream based on an empirically determined constant α, an input video data stream data point at time k x k , a mean of x k at time k μ k , and variance of x k at time k σ k 2 .

3. The method of claim 2 , further comprising determining the moving object in the stabilized video data stream based on determining eccentricity ε k and comparing eccentricity ε k to a constant proportional to α.

4. The method of claim 2 , further comprising determining eccentricity ε k based on recursively updating mean μ k based on a previous mean μ k-1 and weighted video data stream data points αx k that assign decreasing weights to older video data stream data points x k .

5. The method of claim 2 , further comprising determining eccentricity ε k based on recursively updating variance σ k 2 for video data stream data points based on the constant α, the input video data stream data point x k , and the mean μ k .

6. The method of claim 1 , wherein operating the vehicle based on determining at least one moving object in the video data stream includes determining a 3D location of the moving object with respect to the vehicle.

7. The method of claim 6 , wherein the video data stream is acquired by a stationary video camera and determining the 3D location of the moving object with respect to the vehicle includes determining a location and a direction of the stationary video camera with respect to the vehicle.

8. The method of claim 7 , wherein determining location and a direction of the stationary video camera with respect to the vehicle includes determining a 3D pose of the stationary video camera.

9. A system, comprising a processor; and

a memory, the memory including instructions to be executed by the processor to:

receive a video data stream, wherein the video data stream is drift-corrected based on determining that an input video data stream is stable based on determining an eccentricity ε k at a time k and comparing the eccentricity ε k to an empirically determined threshold value, where eccentricity ε k measures a rate at which data points associated with a pixel location x k are changing as a function of time k, and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to real world coordinates; and

operate a vehicle based on determining at least one moving object in a stabilized video data stream.

10. The system of claim 9 , wherein the eccentricity ε k is based on recursively processing the input video data stream based on an empirically determined constant α, an input image stream data point at time k x k , a mean of x k at time k μ k , and variance of x k at time k σ k 2 .

11. The system of claim 10 , the instructions further comprising instructions to determine the moving object in the stabilized video data stream based on determining eccentricity ε k and comparing eccentricity ε k to a constant proportional to α.

12. The system of claim 10 , the instructions further comprising instructions to determine eccentricity ε k based on recursively updating mean μ k based on a previous mean μ k-1 and weighted stationary video data stream data points αx k that assign decreasing weights to older stationary video data stream data points x k .

13. The system of claim 10 , the instructions further comprising instructions to determine eccentricity ε k based on recursively updating variance σ k 2 for stabilized video data stream data points based on the constant α, the input video data stream data point x k , and the mean μ k .

14. The system of claim 9 , wherein operating the vehicle based on determining at least one moving object in the stabilized video data stream includes determining a 3D location of the moving object with respect to the vehicle.

15. The system of claim 14 , wherein the stationary video data stream is acquired by a stationary video camera and determining the 3D location of the moving object with respect to the vehicle includes determining a location and a direction of the stationary video camera with respect to the vehicle.

16. The system of claim 15 , wherein determining location and a direction of the stationary video camera with respect to the vehicle includes determining a 3D pose of the stationary video camera.

17. A system, comprising:

means for controlling vehicle steering, braking, and powertrain; and

computer means for:

receiving a stabilized video data stream, wherein the stabilized video data stream is drift-corrected based on determining that an input video data stream is stable based on determining an eccentricity ε k at a time k and comparing the eccentricity ε k to an empirically determined threshold value, where eccentricity ε k measures a rate at which data points associated with a pixel location x k are changing as a function of time k, and then applying drift correction to maintain a stabilized field of view, wherein the field of view is stabilized with respect to real world coordinates; and

operating a vehicle based on determining at least one moving object in a stabilized image stream and the means for controlling vehicle steering, braking, and powertrain.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2018
From: CORONA, ENRIQUE; GIARDINELLI, STEPHEN; JALES COSTA, BRUNO SIELLY; PARCHAMI, MOSTAFA; PUSKORIUS, GINTARAS VINCENT
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 046608/0533 →
Continuity (1)
Related Publication 20200053286A1 · Feb 13, 2020