Systems and methods for controlling cursor behavior
Systems, methods, and non-transitory computer readable media containing instructions for causing at least one processor to perform operations to enable cursor control in an extended reality space are provided. In one implementation, the processor is configured to perform operations comprising receiving from an image sensor first image data reflecting a first region of focus of a user of a wearable extended reality appliance; causing a first presentation of a virtual cursor in the first region of focus; receiving from the image sensor second image data reflecting a second region of focus of the user outside the initial field of view in the extended reality space; receiving input data indicative of a desire of the user to interact with the virtual cursor; and causing a second presentation of the virtual cursor in the second region of focus in response to the input data.
1. A non-transitory computer readable medium containing instructions for performing operations configured to enable cursor control in an extended reality space, the operations, comprising:
receiving from an image sensor first image data reflecting a first region of focus of a user of a wearable extended reality appliance, wherein the first region of focus is within an initial field of view in an extended reality space;
causing a first presentation of a virtual cursor in the first region of focus;
receiving from the image sensor second image data reflecting a second region of focus of the user outside the initial field of view in the extended reality space;
receiving input data indicative of a desire of the user to interact with the virtual cursor;
causing a second presentation of the virtual cursor in the second region of focus in response to the input data;
estimating a user's awareness of an actual location of the virtual cursor based on the input data and the second image data;
when the estimated user's awareness of the actual location of the virtual cursor is greater than a first threshold, causing a third presentation of the virtual cursor in a manner that follows an intended movement of the virtual cursor; and
when the estimated user's awareness of the actual location of the virtual cursor is below the first threshold, causing the third presentation of the virtual cursor in a manner that does not follow the intended movement of the virtual cursor.
2. The non-transitory computer readable medium of claim 1 ,
estimating the user's awareness of the actual location of the virtual cursor based on the input data and the second image data;
when the estimated user's awareness of the actual location of the virtual cursor is greater than a second threshold, causing a fourth presentation of the virtual cursor in a manner that follows an intended movement of the virtual cursor; and
when the estimated user's awareness of the actual location of the virtual cursor is below the second threshold, causing the fourth presentation of the virtual cursor in a manner that accounts for the intended movement of the virtual cursor.
3. The non-transitory computer readable medium of claim 1 , wherein the operations further include determining that the user lost view of the virtual cursor based on at least one of the input data or the second image data.
4. The non-transitory computer readable medium of claim 1 , wherein the input data is received from a pointing device connected to the wearable extended reality appliance.
5. The non-transitory computer readable medium of claim 4 , wherein the operations further include analyzing the input data to identify a motion pattern indicating that the user lost view of the virtual cursor.
6. The non-transitory computer readable medium of claim 1 , wherein the operations further include determining a discrepancy between a gaze location and a virtual cursor location based on the second image data.
7. The non-transitory computer readable medium of claim 6 , wherein the operations further include:
maintaining a virtual spring between the virtual cursor location and the gaze location; and
activating the virtual spring to bring the virtual cursor location towards the gaze location upon receiving the input data.
8. The non-transitory computer readable medium of claim 7 , wherein the operations further include avoiding activation of the virtual spring when a certainty level of the input data is below a threshold.
9. The non-transitory computer readable medium of claim 1 , wherein the first region of focus is associated with a first virtual object and the second region of focus is associated with a second virtual object, and wherein the operations further include determining a user intent to interact with the second virtual object based on at least one of the input data and the second image data.
10. The non-transitory computer readable medium of claim 1 , wherein the first region of focus is associated with a first virtual window and the second region of focus is associated with a second virtual window, and wherein the operations further include:
analyzing the input data to identify an intention to move the virtual cursor outside a boundary of the first window; and
keeping the virtual cursor within the first window until the user intent is identified.
11. The non-transitory computer readable medium of claim 10 , wherein identifying the user intent to move the virtual cursor outside the boundary of the first window is based on predetermined rules associated with motion of the virtual cursor.
12. The non-transitory computer readable medium of claim 10 , wherein the operations further include causing membrane-like behavior at the boundary of the first window to indicate that the virtual cursor is trying to move outside the boundary of the first window.
13. The non-transitory computer readable medium of claim 1 , wherein causing the second presentation of the virtual cursor includes removing the virtual cursor from the first region of focus and causing the virtual cursor to appear in the second region of focus.
14. The non-transitory computer readable medium of claim 1 , wherein the operations further include:
changing the first presentation of the virtual cursor in the first region of focus in response to the input data, wherein changing the first presentation includes changing a location of the virtual cursor in the first region of focus.
15. The non-transitory computer readable medium of claim 1 , wherein the operations further include:
changing the first presentation of the virtual cursor in the first region of focus in response to the input data, and wherein changing the first presentation includes changing a visual appearance of the virtual cursor.
16. The non-transitory computer readable medium of claim 1 , wherein estimating the user's awareness of the actual location of the virtual cursor includes using a statistical model to learn from previous events that the user lost view of the virtual cursor.
17. The non-transitory computer readable medium of claim 1 , wherein the first image data includes at least one image of a first physical area corresponding to the initial field of view and the second image data includes at least one image of a second physical area corresponding to a second field of view in the extended reality space.
18. A system for cursor control in an extended reality space, the system comprising:
at least one processor configured to:
receive from an image sensor first image data reflecting a first region of focus of a user of a wearable extended reality appliance, wherein the first region of focus is within an initial field of view in an extended reality space;
cause a first presentation of a virtual cursor in the first region of focus;
receive from the image sensor second image data reflecting a second region of focus of the user outside the initial field of view in the extended reality space;
receive input data indicative of a desire of the user to interact with the virtual cursor;
cause a second presentation of the virtual cursor in the second region of focus in response to the input data;
estimate a user's awareness of an actual location of the virtual cursor based on the input data and the second image data;
when the estimated user's awareness of the actual location of the virtual cursor is greater than a first threshold, cause a third presentation of the virtual cursor in a manner that follows an intended movement of the virtual cursor; and
when the estimated user's awareness of the actual location of the virtual cursor is below the first threshold, cause the third presentation of the virtual cursor in a manner that does not follow the intended movement of the virtual cursor.
19. A method for cursor control in an extended reality space, the method comprising:
receiving from an image sensor first image data reflecting a first region of focus of a user of a wearable extended reality appliance, wherein the first region of focus is within an initial field of view in an extended reality space;
causing a first presentation of a virtual cursor in the first region of focus;
receiving from the image sensor second image data reflecting a second region of focus of the user outside the initial field of view in the extended reality space;
receiving input data indicative of a desire of the user to interact with the virtual cursor;
causing a second presentation of the virtual cursor in the second region of focus in response to the input data;
estimating a user's awareness of an actual location of the virtual cursor based on the input data and the second image data;
when the estimated user's awareness of the actual location of the virtual cursor is greater than a first threshold, causing a third presentation of the virtual cursor in a manner that follows an intended movement of the virtual cursor; and
when the estimated user's awareness of the actual location of the virtual cursor is below the first threshold, causing the third presentation of the virtual cursor in a manner that does not follow the intended movement of the virtual cursor.