Smooth object correction for augmented reality devices
An augmented reality (“AR”) device applies smooth correction methods to correct the location of the virtual objects presented to a user. The AR device may apply an angular threshold to determine whether a virtual object can be moved from an original location to a target location. An angular threshold is a maximum angle by which a line from the AR device to the virtual object can change within a timestep. Similarly, the AR device may apply a motion threshold, which is a maximum on the distance that a virtual object's location can be corrected based on the motion of the virtual object. Furthermore, the AR device may apply a pixel threshold to the correction of the virtual object's location. A pixel threshold is a maximum on the distance that a pixel projection of the virtual object can change based on the virtual object's change in location.
1. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
receiving sensor data from one or more sensors of an augmented reality device;
determining a target location for a virtual object, wherein the target location for the virtual object comprises a real-world location that the augmented reality device targets the virtual object to appear to be located when viewed through a display of the augmented reality device;
displaying the virtual object at an original location to a user of the augmented reality device based on the target location and the received sensor data;
receiving updated sensor data from the one or more sensors of the augmented reality device;
determining whether the target location is within a pixel threshold of the original location based on the updated sensor data, wherein the pixel threshold is a maximum number of pixels that the displayed virtual object may move when updating a location of the virtual object; and
responsive to the target location being outside of the pixel threshold of the original location, displaying the virtual object to the user at an updated location, wherein the updated location is within the pixel threshold of the original location and is a closest location to the target location.
2. The computer-readable medium of claim 1 , wherein the one or more sensors comprise one or more of a movement sensor, an accelerometer, a gyroscope, an inertial measurement unit, a barometer, a positioning system, a thermometer, or a light sensor.
3. The computer-readable medium of claim 1 , wherein the operations further comprise:
responsive to the target location being within the pixel threshold of the original location, displaying the virtual object to the user at the target location.
4. The computer-readable medium of claim 1 , wherein determining whether the target location is within a pixel threshold of the original location comprises:
generating a first pixel projection of the virtual object at the original location;
generating a second pixel projection of the virtual object at the target location; and
computing a number of pixels between the first pixel projection and the second pixel projection.
5. The computer-readable medium of claim 4 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first subset of pixels within the first pixel projection that correspond to a portion of the virtual object;
identifying a second subset of pixels within the second pixel projection that correspond to the portion of the virtual object; and
comparing the first subset of pixels to the second subset of pixels.
6. The computer-readable medium of claim 4 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first subset of pixels of the first pixel projection and a second subset of pixels of the second projection, wherein the first subset of pixels and the second subset of pixels are each a subset of pixels of the corresponding pixel projection that is closest to the other pixel projection.
7. The computer-readable medium of claim 4 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first subset of pixels of the first pixel projection and a second subset of pixels of the second projection, wherein the first subset of pixels and the second subset of pixels are each a subset of pixels of the corresponding pixel projection that is furthest from the other pixel projection.
8. The computer-readable medium of claim 4 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first center pixel within the first pixel projection;
identifying a second center pixel within the second pixel projection; and
computing a distance between the first center pixel and the second center pixel.
9. The computer-readable medium of claim 1 , wherein the augmented reality device comprises a smartphone.
10. The computer-readable medium of claim 1 , wherein the augmented reality device comprises a headset.
11. A method comprising:
receiving sensor data from one or more sensors of an augmented reality device;
determining a target location for a virtual object, wherein the target location for the virtual object comprises a real-world location that the augmented reality device targets the virtual object to appear to be located when viewed through a display of the augmented reality device;
displaying the virtual object at an original location to a user of the augmented reality device based on the target location and the received sensor data;
receiving updated sensor data from the one or more sensors of the augmented reality device;
determining whether the target location is within a pixel threshold of the original location based on the updated sensor data, wherein the pixel threshold is a maximum number of pixels that the displayed virtual object may move when updating a location of the virtual object; and
responsive to the target location being outside of the pixel threshold of the original location, displaying the virtual object to the user at an updated location, wherein the updated location is within the pixel threshold of the original location and is a closest location to the target location.
12. The method of claim 11 , wherein the one or more sensors comprise one or more of a movement sensor, an accelerometer, a gyroscope, an inertial measurement unit, a barometer, a positioning system, a thermometer, or a light sensor.
13. The method of claim 11 , further comprising:
responsive to the target location being within the pixel threshold of the original location, displaying the virtual object to the user at the target location.
14. The method of claim 11 , wherein determining whether the target location is within a pixel threshold of the original location comprises:
generating a first pixel projection of the virtual object at the original location;
generating a second pixel projection of the virtual object at the target location; and
computing a number of pixels between the first pixel projection and the second pixel projection.
15. The method of claim 14 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first subset of pixels within the first pixel projection that correspond to a portion of the virtual object;
identifying a second subset of pixels within the second pixel projection that correspond to the portion of the virtual object; and
comparing the first subset of pixels to the second subset of pixels.
16. The method of claim 14 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first subset of pixels of the first pixel projection and a second subset of pixels of the second projection, wherein the first subset of pixels and the second subset of pixels are each a subset of pixels of the corresponding pixel projection that is closest to the other pixel projection.
17. The method of claim 14 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first subset of pixels of the first pixel projection and a second subset of pixels of the second projection, wherein the first subset of pixels and the second subset of pixels are each a subset of pixels of the corresponding pixel projection that is furthest from the other pixel projection.
18. The method of claim 14 , wherein computing a number of pixels between the first pixel projection and the second pixel projection comprises:
identifying a first center pixel within the first pixel projection;
identifying a second center pixel within the second pixel projection; and
computing a distance between the first center pixel and the second center pixel.
19. The method of claim 11 , wherein the augmented reality device comprises a smartphone.
20. The method of claim 11 , wherein the augmented reality device comprises a headset.