Camera motion estimation
View Patent ↗Implementations generally relate to determining camera motion. In one implementation, a method includes capturing a first image and a second image of a physical scene with a camera in respective first and second positions. The method further includes determining first and second image points from the respective first and second image points. The method further includes determining first and second directions of gravity relative to the camera in the respective first and second positions. The method further includes determining a motion of the camera between the capturing of the first image and the capturing of the second image, wherein the determining of the motion of the camera is based at least in part on the first image points, the second image points, the first direction of gravity, and the second direction of gravity.
1. A system comprising:
one or more processors;
logic encoded in one or more non-transitory computer-readable storage media for execution by the one or more processors and when executed operable to perform operations comprising:
capturing a first image of a physical scene with a camera in a first position, wherein the physical scene includes physical feature points;
capturing a second image of the physical scene with the camera in a second position, wherein the first position and the second position are different;
determining first image points from the first image;
determining second image points from the second image;
determining a first direction of gravity relative to the camera in the first position;
determining a second direction of gravity relative to the camera in the second position;
generating a virtual camera having a virtual position, wherein the virtual position of the virtual camera is different from a position of the camera;
generating a virtual image of the physical scene based on the virtual position of the virtual camera, wherein the virtual image is different from the first image and the second image; and
determining a motion of the camera between the capturing of the first image and the capturing of the second image, wherein the determining of the motion of the camera is based at least in part on the first image points, the second image points, the first direction of gravity, the second direction of gravity, and the virtual image.
2. The system of claim 1 , wherein the logic when executed is further operable to perform operations comprising:
determining first virtual points from the first image based at least in part on the first image points and the first direction of gravity, wherein the first virtual points correspond to the physical feature points, and wherein virtual positions of the first virtual points are different from positions of the first image points; and
determining second virtual points from the second image based at least in part on the second image points and the second direction of gravity, wherein the second virtual points correspond to the physical feature points, and wherein virtual positions of the first virtual points are different from positions of the first image points.
3. The system of claim 1 , wherein the logic when executed is further operable to perform operations comprising:
determining first virtual points from the first image;
determining second virtual points from the second image;
comparing the first virtual points to the second virtual points;
determining matched first virtual points and second virtual points; and
selecting at most two matched first virtual points and second virtual points.
4. The system of claim 1 , wherein the logic when executed is further operable to perform operations comprising:
determining first virtual points from the first image;
determining second virtual points from the second image;
comparing the first virtual points to the second virtual points;
determining matched first virtual points and second virtual points, wherein the matched first virtual points and second virtual points are on a same plane such that the plane is perpendicular to a direction of gravity.
5. The system of claim 1 , wherein, to determine the motion of the camera, the logic when executed is further operable to perform operations comprising:
generating a virtual image of the physical scene and the physical feature points and a direction of gravity; and
determining the motion of the camera based at least in part on the virtual image.
6. The system of claim 1 , wherein a line of sight of the virtual camera is perpendicular to a direction of gravity.
7. The system of claim 1 , wherein the determining of the motion of the camera is based on changes in parameter values associated with four degrees of freedom between the first position of the camera when capturing the first image and the second position of the camera when capturing the second image.
8. A non-transitory computer-readable storage medium carrying program instructions thereon, the instructions when executed by one or more processors are operable to perform operations comprising:
capturing a first image of a physical scene with a camera in a first position, wherein the physical scene includes physical feature points;
capturing a second image of the physical scene with the camera in a second position, wherein the first position and the second position are different;
determining first image points from the first image;
determining second image points from the second image;
determining a first direction of gravity relative to the camera in the first position;
determining a second direction of gravity relative to the camera in the second position; and
generating a virtual camera having a virtual position, wherein the virtual position of the virtual camera is different from a position of the camera;
generating a virtual image of the physical scene based on the virtual position of the virtual camera, wherein the virtual image is different from the first image and the second image; and
determining a motion of the camera between the capturing of the first image and the capturing of the second image, wherein the determining of the motion of the camera is based at least in part on the first image points, the second image points, the first direction of gravity, the second direction of gravity, and the virtual image.
9. The computer-readable storage medium of claim 8 , wherein the instructions when executed is further operable to perform operations comprising:
determining first virtual points from the first image based at least in part on the first image points and the first direction of gravity, wherein the first virtual points correspond to the physical feature points; and
determining second virtual points from the second image based at least in part on the second image points and the second direction of gravity, wherein the second virtual points correspond to the physical feature points.
10. The computer-readable storage medium of claim 8 , wherein the instructions when executed is further operable to perform operations comprising:
determining first virtual points from the first image;
determining second virtual points from the second image;
comparing the first virtual points to the second virtual points;
determining matched first virtual points and second virtual points; and
selecting at most two matched first virtual points and second virtual points.
11. The computer-readable storage medium of claim 8 , wherein the instructions when executed are further operable to perform operations comprising:
determining first virtual points from the first image;
determining second virtual points from the second image;
comparing the first virtual points to the second virtual points;
determining matched first virtual points and second virtual points, wherein the matched first virtual points and second virtual points are on a same plane such that the plane is perpendicular to a direction of gravity.
12. The computer-readable storage medium of claim 8 , wherein, to determine the motion of the camera, the instructions when executed are further operable to perform operations comprising:
generating a virtual image of the physical scene and the physical feature points and a direction of gravity; and
determining the motion of the camera based at least in part on the virtual image.
13. The computer-readable storage medium of claim 8 , wherein the instructions when executed are further operable to perform operations generating a virtual image of the physical scene and the physical feature points, and wherein a line of sight of the virtual image is perpendicular to a direction of gravity.
14. The computer-readable storage medium of claim 8 , wherein the determining of the motion of the camera is based on changes in parameter values associated with four degrees of freedom between the first position of the camera when capturing the first image and the second position of the camera when capturing the second image.
15. A method comprising:
capturing a first image of a physical scene with a camera in a first position, wherein the physical scene includes physical feature points;
capturing a second image of the physical scene with the camera in a second position, wherein the first position and the second position are different;
determining first image points from the first image;
determining second image points from the second image;
determining a first direction of gravity relative to the camera in the first position;
determining a second direction of gravity relative to the camera in the second position; and
generating a virtual camera having a virtual position, wherein the virtual position of the virtual camera is different from a position of the camera;
generating a virtual image of the physical scene based on the virtual position of the virtual camera, wherein the virtual image is different from the first image and the second image; and
determining a motion of the camera between the capturing of the first image and the capturing of the second image, wherein the determining of the motion of the camera is based at least in part on the first image points, the second image points, the first direction of gravity, the second direction of gravity, and the virtual image.
16. The method of claim 15 , wherein the method further comprises:
determining first virtual points from the first image based at least in part on the first image points and the first direction of gravity, wherein the first virtual points correspond to the physical feature points; and
determining second virtual points from the second image based at least in part on the second image points and the second direction of gravity, wherein the second virtual points correspond to the physical feature points.
17. The method of claim 15 , wherein the method further comprises:
determining first virtual points from the first image;
determining second virtual points from the second image;
comparing the first virtual points to the second virtual points;
determining matched first virtual points and second virtual points; and
selecting at most two matched first virtual points and second virtual points.
18. The method of claim 15 , wherein the method further comprises determining matched first virtual points and second virtual points, wherein the matched first virtual points and second virtual points are on a same plane such that the plane is perpendicular to a direction of gravity.
19. The method of claim 15 , wherein, to determine the motion of the camera, the method further comprises:
generating a virtual image of the physical scene and the physical feature points and a direction of gravity; and
determining the motion of the camera based at least in part on the virtual image.
20. The method of claim 15 , wherein the method further comprises generating a virtual image of the physical scene and the physical feature points, and wherein a line of sight of the virtual image is perpendicular to a direction of gravity.