Pose estimation with limited correspondences
Various implementations disclosed herein include devices, systems, and methods for pose estimation using one point correspondence, one line correspondence, and a directional measurement. In various implementations, a device includes a non-transitory memory and one or more processors coupled with the non-transitory memory. In some implementations, a method includes obtaining an image corresponding to a physical environment. A first correspondence between a first set of pixels in the image and a spatial point in the physical environment is determined. A second correspondence between a second set of pixels in the image and a spatial line in the physical environment is determined. Pose information is generated as a function of the first correspondence, the second correspondence, and a directional measurement.
1. A method comprising:
at a device including a non-transitory memory and one or more processors coupled with the non-transitory memory:
obtaining an image corresponding to a physical environment;
determining a correspondence between a set of pixels in the image and a spatial point in the physical environment; and
generating pose information as a function of the correspondence and a directional measurement.
2. The method of claim 1 , further comprising determining the directional measurement.
3. The method of claim 2 , further comprising determining the directional measurement based on the image.
4. The method of claim 3 , further comprising determining the directional measurement based on another set of pixels in the image that corresponds to a vertical spatial line in the physical environment.
5. The method of claim 2 , wherein the device comprises an inertial measurement unit (IMU), and the method further comprises obtaining the directional measurement from the IMU.
6. The method of claim 1 , wherein the directional measurement is a gravity vector indicating a direction of gravity, and the method further comprises obtaining the gravity vector.
7. The method of claim 6 , wherein obtaining the gravity vector comprises:
determining a gravity measurement; and
synthesizing the gravity vector based on the gravity measurement.
8. The method of claim 6 , wherein obtaining the gravity vector comprises determining the gravity vector based on an orientation of a physical article that is represented by another set of pixels in the image.
9. The method of claim 1 , further comprising determining another correspondence between another set of pixels in the image and a spatial line in the physical environment; and
wherein generating the pose information comprises generating the pose information as a function of the correspondence, the other correspondence and the directional measurement in order to reduce a number of rotational errors.
10. The method of claim 9 , wherein the correspondence is a point correspondence and the other correspondence is a line correspondence.
11. The method of claim 9 , wherein generating the pose information as the function of the correspondence, the other correspondence, and the directional measurement comprises estimating the pose information.
12. The method of claim 9 , wherein determining the correspondence comprises determining a plurality of first correspondences between a plurality of first sets of pixels in the image and respective spatial points in the physical environment;
wherein determining the other correspondence comprises determining a plurality of second correspondences between a plurality of second sets of pixels in the image and respective spatial lines in the physical environment; and
wherein generating the pose information comprises generating the pose information as a function of the plurality of first correspondences, the plurality of second correspondences, and the directional measurement.
13. The method of claim 12 , wherein generating the pose information comprises generating a plurality of candidate pose estimates as a function of the plurality of first correspondences, the plurality of second correspondences, and the directional measurement.
14. The method of claim 13 , wherein generating the pose information comprises selecting a candidate pose estimate of the plurality of candidate pose estimates that is supported by a threshold number of at least one of the plurality of first correspondences, the plurality of second correspondences, or the directional measurement.
15. A device comprising:
one or more processors;
a non-transitory memory; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
obtain an image corresponding to a physical environment;
determine a correspondence between a set of pixels in the image and a spatial point in the physical environment; and
generate pose information as a function of the correspondence and a directional measurement.
16. The device of claim 15 , wherein the one or more programs further cause the device to determine the directional measurement based on another set of pixels in the image that corresponds to a vertical spatial line in the physical environment.
17. The device of claim 15 , wherein the one or more programs further cause the device to determine another correspondence between another set of pixels in the image and a spatial line in the physical environment; and
wherein generating the pose information comprises generating the pose information as a function of the correspondence, the other correspondence and the directional measurement in order to reduce a number of rotational errors.
18. The device of claim 15 , wherein the directional measurement is a gravity vector indicating a direction of gravity, and the one or more programs further cause the device to obtain the gravity vector.
19. The device of claim 18 , wherein obtaining the gravity vector comprises:
determining a gravity measurement; and
synthesizing the gravity vector based on the gravity measurement.
20. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device, cause the device to:
obtain an image corresponding to a physical environment;
determine a correspondence between a set of pixels in the image and a spatial point in the physical environment; and
generate pose information as a function of the correspondence and a directional measurement.
21. The non-transitory memory of claim 20 , wherein the device comprises an inertial measurement unit (IMU), and the one or more programs further cause the device to obtain the directional measurement from the IMU.
22. The non-transitory memory of claim 20 , wherein the one or more programs further cause the device to determine another correspondence between another set of pixels in the image and a spatial line in the physical environment; and
wherein generating the pose information comprises generating the pose information as a function of the correspondence, the other correspondence and the directional measurement in order to reduce a number of rotational errors.
23. The non-transitory memory of claim 20 , wherein the directional measurement is a gravity vector indicating a direction of gravity, and the one or more programs further cause the device to obtain the gravity vector.
24. The non-transitory memory of claim 23 , wherein obtaining the gravity vector comprises:
determining a gravity measurement; and
synthesizing the gravity vector based on the gravity measurement.
25. The non-transitory memory of claim 23 , wherein obtaining the gravity vector comprises determining the gravity vector based on an orientation of a physical article that is represented by another set of pixels in the image.