IP Library Granted Patent US 9,176,221
Granted Patent B2
US 9,176,221 · App. 13/596,411 · Granted Nov 3, 2015

Distance estimation in camera-based systems utilizing motion measurement and compression attributes

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,176,221
App. No.
13/596,411
Granted
Nov 3, 2015
Kind
B2
Abstract

When estimating distance to an object in an image using a single camera, data acquired by an onboard accelerometer is analyzed to determine camera speed as the user takes a video of the object of interest. The compression that results during video processing is used to derive motion vectors for the object of interest in the scene. The motion vectors have an opposite direction to the camera motion and a magnitude that is a function of the unknown object distance and a known magnification level. The object distance is calculated from the estimated velocity and motion vector magnitude.

Claims (61)

1. A computer-implemented method for estimating distance to an object using motion measurement and video compression attributes derived from video captured by a single camera, comprising:

estimating camera velocity using one or more camera motion values provided by an accelerometer associated with the camera;

identifying frame motion vectors generated during compression of video captured by the camera;

estimating distance to a target object from the one or more camera motion values provided by the accelerometer and motion vector information; and

reporting at least one distance-to-object measurement.

2. The method according to claim 1 wherein the target object distance estimation comprises:

mapping motion vector lengths to block-wise distance values using accelerometer data;

generating a distance-based histogram that represents the relative frequency of occurrence of the block-wise distance values;

identifying peaks in the distance-based histogram in order to isolate objects and remove noise; and

combining distance estimates using variance-based weighting.

3. The method according to claim 1 , wherein the camera resides in a smart phone that has an onboard accelerometer.

4. The method according to claim 1 , wherein the one or more camera motion values represent a direction and a magnitude of camera motion.

5. The method according to claim 1 , wherein the frame motion vectors are determined by searching in the direction of camera motion as determined by analysis of the accelerometer data.

6. The method according to claim 1 , wherein estimating distance to the target object comprises estimating distance as a function of camera motion magnitude and direction of camera motion within a predetermined alignment threshold of the direction of camera motion.

7. The method according to claim 6 , wherein the predetermined alignment threshold is less than or equal to 30° from the direction of camera motion.

8. The method according to claim 6 , wherein estimated camera velocity is described in equation:

mv cam =( d/z )* v real =>z=d ( mv′ cam ·v real )/( mv′ cam ·mv cam )  (4a)

where mv cam is the motion vectors in the camera domain (x′ c i, +y′ c j) and v real is the real velocity (x′i, +y′j).

9. The method according to claim 1 , wherein combining distance estimates using variance-based weighting comprises defining distance estimates such that:

z est ={Σ l ( v′ i-real ·v i-real ) z i }/{Σ l ( v′ i-real ·v i-real )}  (6)

where z i is the distance estimate from the i th frame and v i-real is the measured velocity (in the xy plane—a vector quantity) of the i th frame (average velocity between the i−1th and ith frame).

10. The method according to claim 1 , further comprising reporting error bound information defined by the equation:

z err ≈[z /( v′ real ·v real )]* v real *e vreal +[z 2 /( d *( v′ real ·v real ))]* v real *e mvcam   (5a)

where x′ err and y′ err are the speed error from the accelerometer measurement and x′ cerr and y′ cerr are the errors of the motion vector estimation in the camera domain, for the matrix notion, the vectors e vreal and e mvcam are (x′ err i, y′ err j) and (x′ cerr i, +y′ cerr j) respectively.

11. A processor configured to execute computer-executable instructions for performing the method of claim 1 , the instructions being stored on a non-transitory computer-readable medium.

12. A system that facilitates estimating distance to an object using motion measurement and video compression attributes derived from video captured by a single camera, comprising:

a non-transitory computer-readable medium that stores computer-executable instructions for execution by a processor; and

a processor configured to execute the instructions, the instructions comprising:

estimating camera velocity using one or more camera motion values provided by an accelerometer associated with the camera;

identifying frame motion vectors generated during compression of video captured by the camera;

estimating distance to a target object from the one or more camera motion values provided by the accelerometer and motion vector information; and

reporting at least one distance-to-object measurement.

13. The system according to claim 12 , wherein the target object distance estimation comprises:

mapping motion vector lengths to block-wise distance values using accelerometer data;

generating a distance-based histogram that represents the relative frequency of occurrence of the block-wise distance values;

identifying peaks in the distance-based histogram in order to isolate objects and remove noise; and

combining distance estimates using variance-based weighting.

14. The system according to claim 12 , employed in a smart phone.

15. The system according to claim 12 , wherein the one or more camera motion values represent a direction and a magnitude of camera motion.

16. The system according to claim 12 , wherein the frame motion vectors are determined by searching in the direction of camera motion as determined by analysis of the accelerometer data.

17. The system according to claim 12 , wherein estimating distance to the target object comprises estimating distance as a function of camera motion magnitude and direction of camera motion within a predetermined alignment threshold of the direction of camera motion.

18. The system according to claim 17 , wherein the predetermined alignment threshold is less than or equal to 30° from the direction of camera motion.

19. The system according to claim 17 , wherein estimated camera velocity is described by the equation:

mv cam =( d/z )* v real =>z=d ( mv′ cam ·v real )/( mv′ cam ·mv cam )  (4a)

where mv cam is the motion vectors in the camera domain (x′ c i, +y′ c j) and v real is the real velocity (x′ i, +y′ j).

20. The system according to claim 12 , wherein the instructions for combining distance estimates using variance-based weighting further comprise defining distance estimates such that:

z est ={Σ l ( v′ i-real ·v i-real ) z i }/{Σ l ( v′ i-real ·v i-real )}  (6)

where z i is the distance estimate from the i th frame and v i-real is the measured velocity (in the xy plane—a vector quantity) of the i th frame (average velocity between the i−1th and ith frame).

21. The system according to claim 12 , the instructions further comprising reporting error bound information defined by the equation:

z err ≈[z /( v′ real ·v real )]* v′ real *e vreal +[z 2 /( d *( v′ real ·v real ))]* v′ real *e mvcam   (5a)

where x′ err and y′ err are the speed error from the accelerometer measurement and x′ cerr and y′ cerr are the errors of the motion vector estimation in the camera domain, for the matrix notion, the vectors e vreal and e mvcam are (x′ err i, +y′ err j) and (x′ cerr i, +y′ cerr j) respectively.

22. A non-transitory computer-readable medium having stored thereon computer-executable instructions for estimating distance to an object using motion measurement and video compression attributes derived from video captured by a single camera, the instructions comprising:

estimating camera velocity using one or more camera motion values provided by an accelerometer associated with the camera;

identifying frame motion vectors generated during compression of video captured by the camera;

estimating distance to a target object from the one or more camera motion values provided by the accelerometer and motion vector information; and

reporting at least one distance-to-object measurement.

23. The non-transitory computer-readable medium according to claim 22 , wherein the target object distance estimation comprises:

mapping motion vector lengths to block-wise distance values using accelerometer data;

generating a distance-based histogram that represents the relative frequency of occurrence of the block-wise distance values;

identifying peaks in the distance-based histogram in order to isolate objects and remove noise; and

combining distance estimates using variance-based weighting.

Assignments (4)
SECURITY INTEREST Recorded Oct 19, 2021
From: CONDUENT BUSINESS SERVICES, LLC
To: U.S. BANK, NATIONAL ASSOCIATION
Reel/Frame 057969/0445 →
SECURITY INTEREST Recorded Oct 19, 2021
From: CONDUENT BUSINESS SERVICES, LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 057970/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 28, 2017
From: XEROX CORPORATION
To: CONDUENT BUSINESS SERVICES, LLC
Reel/Frame 041542/0022 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2012
From: SCHWEID, STUART A.; BERNAL, EDGAR A.
To: XEROX CORPORATION
Reel/Frame 028860/0233 →