Method for estimating the speed of movement of a camera
A method for estimating the speed a first video camera when it captures a current image of a three-dimensional scene, the current image including pixels. The method includes storing a reference image corresponding to an image of the same scene captured by a second video camera in a different pose, the reference image including pixels. The method also includes storing the current image, containing for each pixel of the current image the measurement of a physical magnitude measured by that pixel, which is the same as the physical magnitude measured by the pixels of the reference image. The method further includes storing for each pixel of the reference image or of the current image the measurement of a depth that separates that pixel from the point of the scene photographed by that pixel, estimating the pose and speed of the first video camera.
1. A method, comprising:
performing, by a video camera:
capturing a current image of a scene, wherein at least some pixels of the current image are captured in different moments;
obtaining a reference image of the scene corresponding to the current image, wherein all pixels of the reference image are captured in a single moment;
estimating a pose of the video camera;
estimating the speed of the video camera based at least in part on a comparison of reference points in the reference image with corresponding points in the current image, wherein the corresponding points are determined based at least in part on an expected distortion of the video camera at the speed of the video camera; and
storing the estimates for the pose and the speed with the current image.
2. The method of claim 1 , wherein estimating the speed of the video camera comprises:
performing, for individual reference points:
generating a first point in a plane of the current image corresponding to the reference point;
determining a shutter time lag for the first point based at least in part on a pixel location of the first point;
generating a second point in the plane via a shifting of the first point, wherein the shifting reverses a location change of the first point caused by the shutter time lag at an assumed speed of the video camera; and
repeating the generating of respective second points for all reference points, wherein each repeating step updates the assumed speed, and wherein the repeating is stopped when an aggregate difference between magnitude values of the reference points in the reference image and magnitude values of the second points in the current image satisfies a minimization criterion.
3. The method of claim 2 , wherein:
the current image is captured in parallel pixel rows, and successive pixel rows are captured in successive moments; and
determining the shutter time lag for the first point comprises identifying a pixel row to which the first point belongs.
4. The method of claim 2 , wherein the magnitude values of the reference and second points comprise radiation intensity values.
5. The method of claim 2 , wherein the magnitude values of the reference and second points comprise depth values between individual pixels and a physical point in the scene.
6. The method of claim 2 , wherein the estimating of the pose of the video camera is performed in conjunction with the estimating of the speed of the video camera, and the estimate of the pose and the estimate of the speed are updated in conjunction.
7. The method of claim 1 , wherein estimating a pose of the video camera is performed based at least in part on a last estimate for the pose of the video camera and a last estimate for the speed of the video camera.
8. The method of claim 1 , wherein estimating the speed of the video camera comprises estimating three values indicating a translation speed of the video camera and another three values indicating a rotational speed of the video camera, along three mutually orthogonal axes.
9. The method of claim 1 , further comprising:
constructing a trajectory of video camera based at least in part on a plurality of estimates of poses estimated by the video camera.
10. The method of claim 1 , further comprising:
performing, by the video camera:
modifying the current image based at least in part on the estimated speed to correct a distortion caused by movement of the video camera.
11. A system, comprising:
a video camera including one or more processors and associated memory, configured to:
capture a current image of a scene, wherein at least some pixels of the current image are captured in different moments;
obtain a reference image of the scene corresponding to the current image, wherein all pixels of the reference image are captured in a single moment;
estimate a pose of the video camera;
estimate the speed of the video camera based at least in part on a comparison of reference points in the reference image with corresponding points in the current image, wherein the corresponding points are determined based at least in part on an expected distortion of the video camera at the speed of the video camera; and
store the estimates for the pose and the speed with the current image.
12. The system of claim 11 , wherein to estimate the speed of the video camera, the video camera is configured to:
perform, for individual reference points:
generate a first point in a plane of the current image corresponding to the reference point;
determine a shutter time lag for the first point based at least in part on a pixel location of the first point;
generate a second point in the plane via a shifting of the first point, wherein the shifting reverses a location change of the first point caused by the shutter time lag at an assumed speed of the video camera; and
repeat the generating of respective second points for all reference points, wherein each repeating step updates the assumed speed, and wherein the repeating is stopped when an aggregate difference between magnitude values of the reference points in the reference image and magnitude values of the second points in the current image satisfies a minimization criterion.
13. The system of claim 12 , wherein:
the video camera captures the current image in parallel pixel rows, and successive pixel rows are captured in successive moments; and
to determine the shutter time lag for the first point, the video camera identifies a pixel row to which the first point belongs.
14. The system of claim 11 , wherein the video camera is configured to:
construct a trajectory of video camera based at least in part on a plurality of estimates of poses estimated by the video camera.
15. The system of claim 11 , wherein the video camera is configured to:
modify the current image based at least in part on the estimated speed to correct a distortion caused by movement of the video camera.
16. A non-transitory computer readable medium having stored thereon instructions executable by a video camera that when executed causes the video camera to:
capture a current image of a scene, wherein at least some pixels of the current image are captured in different moments;
obtain a reference image of the scene corresponding to the current image, wherein all pixels of the reference image are captured in a single moment;
estimate a pose of the video camera;
estimate the speed of the video camera based at least in part on a comparison of reference points in the reference image with corresponding points in the current image, wherein the corresponding points determined are based at least in part on an expected distortion of the video camera at the speed of the video camera; and
store the estimates for the pose and the speed with the current image.
17. The non-transitory computer readable medium of claim 16 , wherein to estimate the speed of the video camera, the instructions when executed causes the video camera to:
perform, for individual reference points:
generate a first point in a plane of the current image corresponding to the reference point;
determine a shutter time lag for the first point based at least in part on a pixel location of the first point;
generate a second point in the plane via a shifting of the first point, wherein the shifting reverses a location change of the first point caused by the shutter time lag at an assumed speed of the video camera; and
repeat the generating of respective second points for all reference points, wherein each repeating step updates the assumed speed, and wherein the repeating is stopped when an aggregate difference between magnitude values of the reference points in the reference image and magnitude values of the second points in the current image satisfies a minimization criterion.
18. The non-transitory computer readable medium of claim 17 , wherein the magnitude values of the reference and second points comprise depth values between individual pixels and a physical point in the scene.
19. The non-transitory computer readable medium of claim 16 , wherein the instructions when executed causes the video camera to:
construct a trajectory of video camera based at least in part on a plurality of estimates of poses estimated by the video camera.
20. The non-transitory computer readable medium of claim 16 , wherein the instructions when executed causes the video camera to:
modify the current image based at least in part on the estimated speed to correct a distortion caused by movement of the video camera.