Virtual content generation

1. An apparatus for generating virtual content, comprising:

at least one memory; and

at least one processor configured in circuitry and coupled to the at least one memory, the at least one processor being configured to:

obtain a first image of a scene of a real-world environment, wherein the real-world environment is viewable by a user of the apparatus as virtual content is displayed by a display;

detect at least a part of a physical hand of the user in the first image;

determine a first position for a graphical user interface (GUI) element on the display, the first position being determined relative to at least the part of the physical hand of the user;

display the GUI element at the first position on the display;

determine at least the part of the physical hand of the user is in a different location in a second image of the scene as compared to a location of at least the part of the physical hand of the user in the first image; and

display, based on determining at least the part of the physical hand of the user is in the different location in the second image of the scene, the GUI element at a second position on the display, the second position being different than the first position.

2. The apparatus of claim 1 , wherein the GUI element is at least a portion of a virtual keyboard.

3. The apparatus of claim 1 , wherein the at least one processor is configured to:

detect one or more landmark points on the physical hand of the user;

determine one or more locations of the one or more landmark points with respect to a camera used to capture the first image; and

determine, based on the one or more locations of the one or more landmark points with respect to the camera, the first position for the GUI element on the display relative to the physical hand of the user.

4. The apparatus of claim 1 , wherein the at least one processor is configured to:

determine a pose of a head of the user; and

determine, based on the pose of the head, the first position for the GUI element on the display relative to the head.

5. The apparatus of claim 1 , wherein the GUI element is fixed at the first position on the display for at least a period of time as the physical hand of the user moves positions.

6. The apparatus of claim 1 , wherein the at least one processor is configured to:

receive input associated with operation of the GUI element; and

maintain the GUI element at the first position as the GUI element is operated based on the received input.

7. The apparatus of claim 1 , wherein the at least one processor is configured to:

detect expiration of a predetermined amount of time after determining the first position for the GUI element on the display; and

display, based on detecting expiration of the predetermined amount of time, the GUI element at the second position on the display.

8. The apparatus of claim 1 , wherein at least the part of the physical hand of the user includes at least one point on the physical hand of the user and at least one point on a finger of the physical hand of the user.

9. The apparatus of claim 1 , wherein at least the part of the physical hand of the user includes at least one point on the physical hand of the user and at least one point on each finger of the physical hand of the user.

10. The apparatus of claim 1 , wherein the GUI element includes a first part and a second part, the first part being displayed at the first position on the display relative to the physical hand of the user, and the second part being displayed at the second position or a third position on the display relative to an additional physical hand of the user.

11. The apparatus of claim 10 , wherein the first part of the GUI element moves on the display relative to the physical hand of the user, and wherein the second part of the GUI element moves on the display relative to the additional physical hand.

12. The apparatus of claim 1 , wherein the at least one processor is configured to:

determine the physical hand of the user is not present in a third image of the scene; and

remove, based on determining the physical hand of the user is not present in the third image of the scene, the GUI element from the display.

13. The apparatus of claim 12 , wherein the GUI element is active for providing input when the GUI element is removed from the display.

14. The apparatus of claim 13 , wherein the at least one processor is configured to:

deactivate the GUI element from being used for providing input.

15. The apparatus of claim 1 , wherein the at least one processor is configured to:

generate the GUI element in response to detecting at least the part of the physical hand of the user.

16. The apparatus of claim 1 , wherein the real-world environment is viewable through the display by the user as the virtual content is displayed by the display.

17. The apparatus of claim 1 , wherein the apparatus comprises an extended reality device including the display and at least one camera configured to capture one or more images of the scene.

18. A method of generating virtual content, the method comprising:

obtaining, by an extended reality device, a first image of a scene of a real-world environment, wherein the real-world environment is viewable by a user of the extended reality device as virtual content is displayed by a display;

detecting, by the extended reality device, at least a part of a physical hand of the user in the first image;

determining a first position for a graphical user interface (GUI) element on the display, the first position being determined relative to at least the part of the physical hand of the user;

displaying, by the extended reality device, the GUI element at the first position on the display;

determining at least the part of the physical hand of the user is in a different location in a second image of the scene as compared to a location of at least the part of the physical hand of the user in the first image; and

displaying, based on determining at least the part of the physical hand of the user is in the different location in the second image of the scene, the GUI element at a second position on the display, the second position being different than the first position.

19. The method of claim 18 , wherein the GUI element is at least a portion of a virtual keyboard.

20. The method of claim 18 , further comprising:

detecting one or more landmark points on the physical hand of the user;

determining one or more locations of the one or more landmark points with respect to a camera used to capture the first image; and

determining, based on the one or more locations of the one or more landmark points with respect to the camera, the first position for the GUI element on the display relative to the physical hand of the user.

21. The method of claim 18 , further comprising:

determining a pose of a head of the user; and

determining, based on the pose of the head, the first position for the GUI element on the display relative to the head.

22. The method of claim 18 , wherein the GUI element is fixed at the first position on the display for at least a period of time as the physical hand of the user moves positions.

23. The method of claim 18 , further comprising:

receiving input associated with operation of the GUI element; and

maintaining the GUI element at the first position as the GUI element is operated based on the received input.

24. The method of claim 18 , further comprising:

detecting expiration of a predetermined amount of time after determining the first position for the GUI element on the display; and

displaying, based on detecting expiration of the predetermined amount of time, the GUI element at the second position on the display.

25. The method of claim 18 , wherein at least the part of the physical hand of the user includes at least one point on the physical hand of the user and at least one point on a finger of the physical hand of the user.

26. The method of claim 18 , wherein at least the part of the physical hand of the user includes at least one point on the physical hand of the user and at least one point on each finger of the physical hand of the user.

27. The method of claim 18 , wherein the GUI element includes a first part and a second part, the first part being displayed at the first position on the display relative to the physical hand of the user, and the second part being displayed at the second position or a third position on the display relative to an additional physical hand of the user.

28. The method of claim 18 , further comprising:

determining the physical hand of the user is not present in a third image of the scene; and

removing, based on determining the physical hand of the user is not present in the third image of the scene, the GUI element from the display.

29. The method of claim 18 , further comprising:

generating the GUI element in response to detecting at least the part of the physical hand of the user.

30. The method of claim 18 , wherein the real-world environment is viewable through the display by the user as the virtual content is displayed by the display.