Methods and systems of extended reality environment interaction based on eye motions
Systems and methods are described for extended reality environment interaction. An extended reality environment including an object is generated for display, and an eye motion is detected. Based on the detecting, it is determined whether the object is in a field of view for at least a predetermined period of time, and in response to determining that the object is in the field of view for at least the predetermined period of time, one or more items related to the object are generated for display in the extended virtual reality environment.
1. A computer-implemented method for extended reality environment interaction comprising:
determining whether a gaze of a user is directed at a first object in the extended reality environment for at least a particular period of time;
in response to determining that the gaze of the user is directed at the first object for at least the particular period of time, generating for display in the extended reality environment one or more items related to the first object;
determining that the gaze of the user has shifted from the first object to a second object in the extended reality environment; and
in response to determining that the gaze of the user has remained directed at the second object for at least the particular period of time, without the gaze of the user shifting back to the first object, generating for display in the extended reality environment one or more items related to the second object, while continuing to generate for display in the extended reality environment, the one or more items related to the first object.
2. The computer-implemented method of claim 1 , wherein generating for display in the extended reality environment the one or more items related to the second object, while continuing to generate for display in the extended reality environment the one or more items related to the first object, is further performed in response to determining that each of the first object and the second object is in a field of view of the user.
3. The computer-implemented method of claim 1 , wherein:
generating for display in the extended reality environment the one or more items related to the second object, while continuing to generate for display in the extended reality environment the one or more items related to the first object, is performed in a first extended reality session at a first time;
determining that a request has been received from the user to initiate a second extended reality session at a second time later than the first time; and
in response to initiating the second extended reality session, causing at least one of the one or more items related to the first object, or the one or more items related to the second object, to be generated for display.
4. The computer-implemented method of claim 1 , wherein the determining whether the gaze of the user is directed at the first object in the extended reality environment comprises monitoring an eye motion of the user.
5. The computer-implemented method of claim 1 , further comprising:
generating for display to the user a text-based indication that the one or more items related to the first object will be displayed if the gaze of the user is directed towards the first object.
6. The computer-implemented method of claim 1 , wherein the determining whether the gaze of the user is directed at the first object in the extended reality environment for at least the particular period of time comprises determining that the first object is continuously in a field of view of the user for the particular period of time during an extended reality session.
7. The computer-implemented method of claim 1 , wherein the determining whether the gaze of the user is directed at the first object in the extended reality environment for at least the particular period of time comprises determining that the first object is non-continuously in a field of view of the user for the particular period of time during an extended reality session.
8. The computer-implemented method of claim 1 , further comprising:
determining whether the first object in the extended reality environment is in the gaze of the user by:
accessing a data structure with a plurality of object identifiers associated with a plurality of objects in a field of view of the extended reality environment, wherein a respective position of each object in the field of view in the extended reality environment is represented by coordinates; and
determining that the gaze of the user is directed at the first object, from among the plurality of objects in the extended reality environment, based on sensor data and the coordinates of the first object.
9. The computer-implemented method of claim 8 , wherein the sensor data comprises one or more images captured by a camera.
10. The computer-implemented method of claim 1 , wherein the one or more items related to the second object are generated for display further in response to receiving a voice command requesting that additional information related to the second object be provided.
11. A system comprising:
a display; and
a control circuitry configured to:
determine whether a gaze of a user is directed at a first object in an extended reality environment for at least a particular period of time;
in response to determining that the gaze of the user is directed at the first object for at least the particular period of time, generate for display, at the display, in the extended reality environment, one or more items related to the first object;
determine that the gaze of the user has shifted from the first object to a second object in the extended reality environment; and
in response to determining that the gaze of the user has remained directed at the second object for at least the particular period of time, without the gaze of the user shifting back to the first object, generate for display, at the display, in the extended reality environment, one or more items related to the second object, while continuing to generate for display in the extended reality environment, the one or more items related to the first object.
12. The system of claim 11 , wherein the control circuitry is further configured to generate for display in the extended reality environment the one or more items related to the second object, while continuing to generate for display in the extended reality environment the one or more items related to the first object by determining that each of the first object and the second object is in a field of view of the user.
13. The system of claim 11 , wherein the control circuitry is further configured to generate for display in the extended reality environment the one or more items related to the second object, while continuing to generate for display in the extended reality environment the one or more items related to the first object, in a first extended reality session at a first time by:
determining that a request has been received from the user to initiate a second extended reality session at a second time later than the first time; and
in response to initiating the second extended reality session, causing at least one of the one or more items related to the first object, or the one or more items related to the second object, to be generated for display.
14. The system of claim 11 , wherein the control circuitry is configured to determine whether the gaze of the user is directed at the first object in the extended reality environment by monitoring an eye motion of the user.
15. The system of claim 11 , wherein the control circuitry is further configured to:
generate for display to the user a text-based indication that the one or more items related to the first object will be displayed if the gaze of the user is directed towards the first object.
16. The system of claim 11 , wherein the control circuitry is configured to determine whether the gaze of the user is directed at the first object in the extended reality environment for at least the particular period of time by determining that the first object is continuously in a field of view of the user for the particular period of time during an extended reality session.
17. The system of claim 11 , wherein the control circuitry is configured to determine whether the gaze of the user is directed at the first object in the extended reality environment for at least the particular period of time by determining that the first object is non-continuously in a field of view of the user for the particular period of time during an extended reality session.
18. The system of claim 11 , wherein the control circuitry is further configured to:
determine whether the first object in the extended reality environment is in the gaze of the user by:
accessing a data structure with a plurality of object identifiers associated with a plurality of objects in a field of view of the extended reality environment, wherein a respective position of each object in the field of view in the extended reality environment is represented by coordinates; and
determining that the gaze of the user is directed at the first object, from among the plurality of objects in the extended reality environment, based on sensor data and the coordinates of the first object.
19. The system of claim 18 , wherein the sensor data comprises one or more images captured by a camera.
20. The system of claim 11 , wherein the control circuitry is further configured to generate for display the one or more items related to the second object in response to receiving a voice command requesting that additional information related to the second object be provided.