Tracking in a moving platform

1. A method comprising:

receiving, from an inertial measurement unit on an entity in a moving platform, data sensed by the inertial measurement unit;

receiving, from a sensor on a second inertial measurement unit on the moving platform, visual tracking data computed from images depicting the moving platform or the entity in the moving platform; and

computing a three-dimensional (3D) position and orientation of the entity at least by estimating a plurality of states using the received visual tracking data and the received data sensed by the inertial measurement unit, wherein the plurality of states comprises a state of the moving platform and a state of the entity.

2. The method of claim 1 where the plurality of states of the moving platform comprises one or more of: platform angular acceleration, platform linear acceleration, platform angular velocity, platform rotation in a current coordinate frame used by a visual tracker.

3. The method of claim 1 further comprising using the computed 3D position and orientation of the entity to do one or more of: place a hologram in a field of view of the entity such that the hologram appears locked to the moving platform, control movement of the entity in the moving platform.

4. The method of claim 1 wherein computing the 3D position and orientation of the entity comprises using state propagation and an update equation which take into account a second order effect.

5. The method of claim 1 further comprising:

receiving data sensed by a second inertial measurement unit, wherein the second inertial measurement unit is mounted on the moving platform; and

determining a 3D position and orientation of the second inertial measurement unit.

6. The method of claim 5 wherein determining a 3D position and orientation of the second inertial measurement unit comprises using a visual reference.

7. The method of claim 5 wherein determining a 3D position and orientation of the second inertial measurement unit comprises capturing an image depicting a user gesturing towards the second inertial measurement unit.

8. The method of claim 5 wherein determining a 3D position and orientation of the second inertial measurement unit comprises capturing an image depicting a code, wherein the code visually defines a coordinate system of the second inertial measurement unit.

9. The method of claim 5 further comprising using the data sensed by the second inertial measurement unit and the determined 3D position and orientation of the second inertial measurement unit to compute the 3D position and orientation of the entity.

10. The method of claim 9 wherein the states of the moving platform used in the computation are observed from the data sensed by the second inertial measurement unit and the determined 3D position and orientation of the second inertial measurement unit.

11. The method of claim 5 wherein the states comprise a 3D position and orientation of the second inertial measurement unit expressed in a world reference frame and a velocity of the second inertial measurement unit expressed in the world reference frame.

12. The method of claim 11 further comprising triggering display of a hologram outside the moving platform using the 3D position and orientation, such that the hologram appears locked to an environment outside the moving platform from a view point of the entity.

13. An apparatus comprising:

a processor; and

computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, implement a method comprising:

receiving, from an inertial measurement unit on an entity in a moving platform, data sensed by the inertial measurement unit;

receiving, from a sensor on a second inertial measurement unit on the moving platform, visual tracking data computed from images depicting the moving platform or the entity in the moving platform; and

computing a three-dimensional (3D) position and orientation of the entity by estimating a plurality of states using the received visual tracking data and the received data sensed by the inertial measurement unit, wherein the plurality of states comprises a state of the moving platform and a state of the entity.

14. The apparatus of claim 13 wherein:

the apparatus is a head-mounted display, and

the instructions, when executed by the processor, implement the method to trigger display of a hologram on the basis of the computed 3D position and orientation of the entity.

15. The apparatus of claim 13 wherein:

the apparatus is a robot or unmanned aerial vehicle, and

the instructions, when executed by the processor, implement the method to control movement of the robot or unmanned aerial vehicle.

16. A system comprising:

a mobile computing device having an inertial measurement unit, the mobile computing device being mounted on a moving platform in use;

an apparatus for tracking three-dimensional (3D) position and orientation of an entity in the moving platform, the apparatus comprising:

a processor; and

computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, implement a method comprising:

receiving, from the inertial measurement unit, data sensed by the inertial measurement unit;

receiving, from a sensor on a second inertial measurement unit on the moving platform, visual tracking data computed from images depicting the moving platform or the entity in the moving platform;

determining a position and orientation of the mobile computing device; and

computing the 3D position and orientation of the entity by estimating a plurality of states using the received visual tracking data sensed by the inertial measurement unit and the determined position and orientation of the mobile computing device, wherein the plurality of states comprises a state of the moving platform and a state of the entity.

17. The system of claim 16 wherein determining the position and orientation of the mobile computing device comprises capturing an image of a user gesturing towards the mobile computing device or computing a gaze location of a user looking at the mobile computing device.

18. The system of claim 16 wherein determining the position and orientation of the mobile computing device comprises using a visual reference or a QR code.

19. The system of claim 16 wherein the states comprise (i) a 3D position and orientation of the mobile computing device, and (ii) a velocity of the inertial measurement unit of the mobile computing device.

20. The system of claim 19 wherein the instructions, when executed by the processor, further implement the method comprising triggering display of a hologram outside the moving platform using the tracked 3D position and orientation, such that the hologram appears locked to an environment outside the moving platform from a view point of the entity.