Rerendering a position of a hand to decrease a size of a hand to create a realistic virtual/augmented reality environment
The technology disclosed relates to a method of realistic rendering of a real object as a virtual object in a virtual space using an offset in the position of the hand in a three-dimensional (3D) sensory space. An offset between expected positions of the eye(s) of a wearer of a head mounted device and a sensor attached to the head mounted device for sensing a position of at least one hand in a three-dimensional (3D) sensory space is determined. A position of the hand in the three-dimensional (3D) sensory space can be sensed using a sensor. The sensed position of the hand can be transformed by the offset into a re-rendered position of the hand as would appear to the wearer of the head mounted device if the wearer were looking at the actual hand. The re-rendered hand can be depicted to the wearer of the head mounted device.
1. A system including: a processor; and a memory storing computer instructions that, when executed by the processor, perform operations comprising: determining an offset between an eye of a user and an actual position of a sensor; determining one or more control object properties; obtaining a re-rendered position of a control object, the re-rendered position of the control object being a result of an application of a transformation matrix to a position of the control object determined in dependence on the one or more control object properties, the transformation matrix being based on the actual position of the sensor and for decreasing a rendered size of the control object in accordance with the offset between the eye of the user and the actual position of the sensor; and providing, for display to the user, a control object image using the re-rendered position of the control object.
2. The system of claim 1 , further configured to perform: obtaining additional re-rendered positions of the control object, the additional re-rendered positions being based on the offset being applied to sensed additional positions of the control object, thereby providing the control object image having a motion or gesture of the control object.
3. The system of claim 2 , further configured to perform: obtaining an indication of quality of tracking for the sensor, the indication of quality of tracking indicating whether the sensor is tracking a position or motion of the control object; and adding to the control object image a visual indication corresponding to the indication of quality of tracking, thereby providing the user with a visual cue of how well the sensor is tracking the control object.
4. The system of claim 3 , wherein the indication of quality of tracking includes an indication that the sensor is able to recognize the control object in captured images.
5. The system of claim 3 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the control object image when the sensor is unable to recognize the control object in captured images.
6. The system of claim 1 , wherein the control object image comprises a non-rigid transformation based on an error indication applied after the transformation matrix is applied to the position of the control object.
7. A method comprising: determining an offset between an eye of a user and an actual position of a sensor; determining one or more control object properties; obtaining a re-rendered position of a control object, the re-rendered position of the control object being a result of an application of a transformation matrix to a position of the control object determined in dependence on the one or more control object properties, the transformation matrix being based on the actual position of the sensor and for decreasing a rendered size of the control object in accordance with the offset between the eye of the user and the actual position of the sensor; and providing, for display to the user, a control object image using the re-rendered position of the control object.
8. The method of claim 7 , further including: obtaining additional re-rendered positions of the control object, the additional re-rendered positions being based on the offset being applied to sensed additional positions of the control object, thereby providing the control object image having a motion or gesture of the control object.
9. The method of claim 8 , further including: obtaining an indication of quality of tracking for the sensor, the indication of quality of tracking indicating whether the sensor is tracking a position or motion of the control object; and adding to the control object image a visual indication corresponding to the indication of quality of tracking, thereby providing the user with a visual cue of how well the sensor is tracking the control object.
10. The method of claim 9 , wherein the indication of quality of tracking includes an indication that the sensor is able to recognize the control object in captured images.
11. The method of claim 9 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the control object image when the sensor is unable to recognize the control object in captured images.
12. The method of claim 8 , wherein: the control object image comprises a non-rigid transformation based on an error indication applied after the transformation matrix is applied to the position of the control object.
13. A non-transitory computer-readable recording medium having instructions recorded thereon, the instructions, when executed by a processor, perform a method comprising: determining an offset between an eye of a user and an actual position of a sensor; determining one or more control object properties; obtaining a re-rendered position of a control object, the re-rendered position of the control object being a result of an application of a transformation matrix to a position of the control object determined in dependence on the one or more control object properties, the transformation matrix being based on the actual position of the sensor and for decreasing a rendered size of the control object in accordance with the offset between the eye of the user and the actual position of the sensor; and providing, for display to the user, a control object image using the re-rendered position of the control object.
14. The non-transitory computer-readable recording medium of claim 13 , wherein the method further comprises: obtaining additional re-rendered positions of the control object, the additional re-rendered positions being based on the offset being applied to sensed additional positions of the control object, thereby providing the control object image having a motion or gesture of the control object.
15. The non-transitory computer-readable recording medium of claim 14 , wherein the method further comprises: obtaining an indication of quality of tracking for the sensor, the indication of quality of tracking indicating whether the sensor is tracking a position or motion of the control object; and adding to the control object image a visual indication corresponding to the indication of quality of tracking, thereby providing the user with a visual cue of how well the sensor is tracking the control object.
16. The non-transitory computer-readable recording medium of claim 15 , wherein the indication of quality of tracking includes an indication that the sensor is able to recognize the control object in captured images.
17. The non-transitory computer-readable recording medium of claim 15 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the control object image when the sensor is unable to recognize the control object in captured images.
18. The non-transitory computer-readable recording medium of claim 15 , wherein the visual indication corresponding to the indication of quality of tracking includes coloring the control object image a different color when the sensor is able to recognize the control object in captured images than when the sensor is unable to recognize the control object in captured images.
19. The non-transitory computer-readable recording medium of claim 14 , wherein: the control object image comprises a non-rigid transformation based on an error indication applied after the transformation matrix is applied to the position of the control object.
20. The non-transitory computer-readable recording medium of claim 14 , wherein the method further comprises: sensing an indication of whether the control object has grabbed a virtual object; and adding to the control object image a visual indication corresponding to the indication of whether the control object has grabbed the virtual object, thereby providing the user with a visual cue of whether the virtual object is grabbed.