Visual-based inertial navigation

1. An electronic computing system for performing navigation, the electronic computing system comprising an electronic processor and memory operable to implement a sliding-window inverse filter module, wherein the sliding-window inverse filter module is configured to:

receive sensor measurements from a pre-processing module, wherein the sensor measurements comprise image data and inertial data for a device;

transfer information, derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first and second windows consecutively overlap in time; and

output, to a post-processing module, a state of the device based on the transferred information,

wherein each window in the pair is associated with a corresponding first information matrix, and each first information matrix is for a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and

wherein transferring the information comprises:

marginalizing at least a subset of the first variables associated with the first information matrix of the first window to obtain a second information matrix, which still maintains the association with the plurality of second variables;

marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix; and

scaling the third information matrix to obtain a fourth information matrix,

wherein the fourth information matrix is a summarized representation of the information about the state of the device and of the position and/or orientation of the objects proximate to the device at points in time when the first window and the second window in the pair overlap.

2. The electronic computing system of claim 1 , further comprising:

the pre-processing module;

an image detection unit coupled to the pre-processing module, wherein the pre-processing module is configured to receive a plurality of images captured by the image detection unit and derive the image data from the plurality of images; and

a motion sensing unit coupled to the pre-processing module, wherein the pre-processing module is configured to receive the inertial data from the motion sensing unit.

3. The electronic computing system of claim 1 , wherein a subset of the first set of variables associated with the first window overlaps in time with a subset of the second set of variables associated with the second window.

4. The electronic computing system of claim 3 , wherein the overlap between the first set of variables and the second set of variables is at the earliest time that is present in both the first and second windows.

5. The electronic computing system of claim 1 , wherein, for each window, the plurality of first variables represents 3D positions and/or orientations of image features across a plurality of images, and

wherein, for each window, the plurality of second variables comprises linear acceleration data and angular velocity data.

6. The electronic computing system of claim 5 , wherein the state of the device further comprises biases for the motion sensing unit.

7. The electronic computing system of claim 1 , wherein the first information matrix associated with each window comprises:

first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device; and

second information about at least one variable representing an estimate of the state of the device; and

shared information between the first information and the second information.

8. The electronic computing system of claim 7 , wherein the shared information represents shared confidence between the estimates of a position or orientation of an object proximate to the device and the estimate of the state of the device.

9. The electronic computing system of claim 7 ,

wherein marginalizing at least the subset of the first variables associated with the first information matrix of the first window comprises marginalizing the shared information, and

wherein scaling the third information matrix comprises dividing or multiplying the third information matrix by a constant.

10. The electronic computing system of claim 9 , wherein the constant is equal to a size of the first window.

11. A computer-implemented method comprising:

receiving sensor measurements from a pre-processing module, wherein the sensor measurements comprise image data and inertial data for a device;

transferring, using a processor, information derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first and second windows consecutively overlap in time; and

outputting, to a post-processing module, a state of the device based on the transferred information,

wherein each window in the pair is associated with a corresponding first information matrix, and each first information matrix is for a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and

wherein transferring the information comprises:

marginalizing at least a subset of the first variables associated with the first information matrix of the first window to obtain a second information matrix, which still maintains the association with the plurality of second variables;

marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix; and

scaling the third information matrix to obtain a fourth information matrix,

wherein the fourth information matrix is a summarized representation of the information in the state of the device and of the position and/or orientation of the objects proximate to the device at points in time when the first window and the second window in the pair overlap.

12. The computer-implemented method of claim 11 , wherein a subset of the first set of variables associated with the first window overlaps in time with a subset of the second set of variables associated with the second window.

13. The computer-implemented method of claim 12 , wherein the overlap between the first set of variables and the second set of variables is at the earliest time that is present in both the first and second windows.

14. The computer-implemented method of claim 11 , wherein, for each window, the plurality of first variables represents 3D positions and/or orientations of image features across a plurality of images, and

wherein, for each window, the plurality of second variables comprises linear acceleration data and angular velocity data.

15. The computer-implemented method of claim 14 , wherein the state of the device further comprises biases for the motion sensing unit.

16. The computer-implemented method of claim 11 , wherein the first information matrix associated with each window comprises:

first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device; and

second information about at least one variable representing an estimate of the state of the device; and

shared information between the first information and the second information.

17. The computer-implemented method of claim 16 , wherein the shared information represents shared confidence between the estimates of a position or orientation of an object proximate to the device and the estimate of the state of the device.

18. The computer-implemented method of claim 16 ,

wherein marginalizing at least the subset of the first variables associated with the first information matrix of the first window comprises marginalizing the shared information, and

wherein scaling the third information matrix comprises dividing or multiplying the third information matrix by a constant.

19. The computer-implemented method of claim 18 , wherein the constant is equal to a size of the first window.

20. A non-transitory computer storage medium encoded with computer program instructions that when executed by one or more electronic computing devices cause the one or more electronic computing devices to perform operations comprising:

receiving sensor measurements from a pre-processing module, wherein the sensor measurements comprise image data and inertial data for a device;

transferring, using a processor, information derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first and second windows consecutively overlap in time; and

outputting, to a post-processing module, a state of the device based on the transferred information,

wherein each window in the pair is associated with a corresponding first information matrix, and each first information matrix is for a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and

wherein transferring the information comprises:

marginalizing at least a subset of the first variables associated with the first information matrix of the first window to obtain a second information matrix, which still maintains the association with the plurality of second variables;

marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix; and

scaling the third information matrix to obtain a fourth information matrix,

wherein the fourth information matrix is a summarized representation of the information about the state of the device and of the position and/or orientation of the objects proximate to the device at points in time when the first window and the second window in the pair overlap.

21. The medium of claim 20 , wherein a subset of the first set of variables associated with the first window overlaps in time with a subset of the second set of variables associated with the second window.

22. The medium of claim 21 , wherein the overlap between the first set of variables and the second set of variables is at the earliest time that is present in both the first and second windows.

23. The medium of claim 20 , wherein, for each window, the plurality of first variables represents 3D positions and/or orientations of image features across a plurality of images, and

wherein, for each window, the plurality of second variables comprises linear acceleration data and angular velocity data.

24. The medium of claim 23 , wherein the state of the device further comprises biases for the motion sensing unit.

25. The medium of claim 20 , wherein the first information matrix associated with each window comprises:

first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device; and

second information about at least one variable representing an estimate of the state of the device; and

shared information between the first information and the second information.

26. The medium of claim 25 , wherein the shared information represents shared confidence between the estimates of a position or orientation of an object proximate to the device and the estimate of the state of the device.

27. The medium of claim 25 , wherein marginalizing at least the subset of the first variables associated with the first information matrix of the first window comprises marginalizing the shared information, and

wherein scaling the third information matrix comprises dividing or multiplying the third information matrix by a constant.

28. The medium of claim 27 , wherein the constant is equal to a size of the first window.