Dead reckoning positional prediction for augmented reality and virtual reality applications

1. A method of predicting a virtual camera view in an augmented reality (AR) or virtual reality (VR) application, comprising:

determining a first change in position of a user device over a first time period based on analyzing a plurality of frames of image data related to an AR or VR application, comprising:

identifying a first position of the user device at a first point in time based on analyzing a first image captured using the user device;

identifying a second position of the user device at a second point in time based on analyzing a second image captured using the user device, wherein the second point in time occurs after the first point in time; and

identifying the first change in position based on the identified first position of the user device and the identified second position of the user device;

predicting a second change in position of the user device over a second time period, using a dead reckoning calculation and based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device; and

generating a plurality of frames of image data for display in the AR or VR application, based on the predicted second change in position of the user device.

2. The method of claim 1 , wherein generating the plurality of frames of image data for display in the AR or VR application further comprises:

determining a virtual camera view associated with the AR or VR application, wherein the virtual camera view includes a virtual object and a physical object, wherein a relationship between the virtual object and the physical object in the virtual camera view is based on the predicted second change in position of the user device, and wherein at least one of the plurality of frames of image data is based on the determined virtual camera view.

3. The method of claim 1 , wherein identifying the first position of the user device comprises identifying a first location of a reference object in the first image and wherein identifying the second position of the user device comprises identifying a second location of the reference object in the second image.

4. The method of claim 1 , further comprising:

determining a first change in attitude of the user device over the first time period based on analyzing data received from the IMU associated with the user device, wherein the data relates to the first time period, and wherein predicting the second change in position of the user device is further based on the determined first change in attitude of the user device.

5. The method of claim 1 , wherein generating the plurality of frames of image data for display in the AR or VR application further comprises:

identifying a display frame rate associated with the AR or VR application;

determining a first number of frames for generation, based on the identified display frame rate and the second time period; and

generating the first number of frames of image data.

6. The method of claim 1 , further comprising:

determining a yaw correction value associated with the user device during the first time period based on analyzing data from a magnetometer associated with the user device, wherein the data relates to the first time period, and wherein predicting the second change in position of the user device is further based on the determined yaw correction value.

7. The method of claim 1 , wherein the user device comprises a head mounted display.

8. The method of claim 1 , wherein the application is an AR application.

9. A computer program product for predicting a virtual camera view in an AR or VR application, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation, the operation comprising:

determining a first change in position of a user device over a first time period based on analyzing a plurality of frames of image data related to an AR or VR application, comprising:

identifying a first position of the user device at a first point in time based on analyzing a first image captured using the user device;

identifying a second position of the user device at a second point in time based on analyzing a second image captured using the user device, wherein the second point in time occurs after the first point in time; and

identifying the first change in position based on the identified first position of the user device and the identified second position of the user device;

predicting a second change in position of the user device over a second time period, using a dead reckoning calculation and based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device; and

generating a plurality of frames of image data for display in the AR or VR application, based on the predicted second change in position of the user device.

10. The computer program product of claim 9 , wherein generating the plurality of frames of image data for display in the AR or VR application further comprises:

determining a virtual camera view associated with the AR or VR application, wherein the virtual camera view includes a virtual object and a physical object, wherein a relationship between the virtual object and the physical object in the virtual camera view is based on the predicted second change in position of the user device, and wherein at least one of the plurality of frames of image data is based on the determined virtual camera view.

11. The computer program product of claim 9 , wherein identifying the first position of the user device comprises identifying a first location of a reference object in the first image and wherein identifying the second position of the user device comprises identifying a second location of the reference object in the second image.

12. The computer program product of claim 9 , the operation further comprising:

determining a first change in attitude of the user device over the first time period based on analyzing data received from the IMU associated with the user device, wherein the data relates to the first time period, and wherein predicting the second change in position of the user device is further based on the determined first change in attitude of the user device.

13. The computer program product of claim 9 , wherein generating the plurality of frames of image data for display in the AR or VR application further comprises:

identifying a display frame rate associated with the AR or VR application;

determining a first number of frames for generation, based on the identified display frame rate and the second time period; and

generating the first number of frames of image data.

14. A system, comprising:

a processor; and

a memory storing a program, which, when executed on the processor, performs an operation, the operation comprising:

determining a first change in position of a user device over a first time period based on analyzing a plurality of frames of image data related to an AR or VR application, comprising:

identifying a first position of the user device at a first point in time based on analyzing a first image captured using the user device;

identifying a second position of the user device at a second point in time based on analyzing a second image captured using the user device, wherein the second point in time occurs after the first point in time; and

identifying the first change in position based on the identified first position of the user device and the identified second position of the user device;

predicting a second change in position of the user device over a second time period, using a dead reckoning calculation and based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device; and

generating a plurality of frames of image data for display in the AR or VR application, based on the predicted second change in position of the user device.

15. The system of claim 14 , wherein generating the plurality of frames of image data for display in the AR or VR application further comprises:

determining a virtual camera view associated with the AR or VR application, wherein the virtual camera view includes a virtual object and a physical object, wherein a relationship between the virtual object and the physical object in the virtual camera view is based on the predicted second change in position of the user device, and wherein at least one of the plurality of frames of image data is based on the determined virtual camera view.

16. The system of claim 14 , wherein identifying the first position of the user device comprises identifying a first location of a reference object in the first image and wherein identifying the second position of the user device comprises identifying a second location of the reference object in the second image.

17. The system of claim 14 , the operation further comprising:

determining a first change in attitude of the user device over the first time period based on analyzing data received from the IMU associated with the user device, wherein the data relates to the first time period, and wherein predicting the second change in position of the user device is further based on the determined first change in attitude of the user device.

18. The method of claim 1 , wherein the first point in time and the second point in time each occur in the first time period.