IP Library Granted Patent US 10,055,028
Granted Patent B2
US 10,055,028 · App. 15/369,478 · Granted Aug 21, 2018

End of session detection in an augmented and/or virtual reality environment

Inventor: Jason Toff (Hoboken, NJ)
Assignee: GOOGLE LLC
G06F3/0346G06F3/011G06F3/012G06F3/013G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,055,028
App. No.
15/369,478
Filed
Dec 5, 2016
Granted
Aug 21, 2018
Kind
B2
Art Unit
2687
USPC
345/156
Abstract

In an augmented reality and/or a virtual reality system, termination of a virtual session, and/or suspension of a virtual session, may be detected based on detection of a position and/or orientation of a head mounted display device, and/or movement and position/orientation of the head mounted display device worn by the user. Follow on processing, and/or supplemental tasking, associated with the recently terminated virtual session, may be automatically initiated in response to the detection of the termination and/or suspension of the virtual session, without user intervention and/or manual input of a command instructing the system to initiate processing.

Claims (76)

1. A method, comprising:

generating a virtual environment in a head mounted display (HMD) device operating in an ambient environment;

initiating a virtual session and receiving at least one user input for executing at least one corresponding command in the virtual environment while the virtual session is in an active state;

detecting at least one of a position or an orientation of the HMD in the ambient environment;

detecting a gesture input based on the at least one of the position or the orientation of the HMD;

detecting an end of the virtual session based on the detection of the gesture input; and

initiating, in response to the detected end of the virtual session, processing related to the at least one user input received during the virtual session while in the active state.

2. The method of claim 1 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting an orientation of the HMD;

comparing the detected orientation of the HMD to a set orientation; and

detecting the gesture input corresponding to the detection of the end of the virtual session when, based on the comparison it is determined that the detected orientation of the HMD matches the preset orientation of the HMD for greater than or equal to a set amount of time.

3. The method of claim 2 , wherein the set orientation of the HMD corresponds to a horizontal alignment of a centerline of the HMD worn by the user.

4. The method of claim 1 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting the position and the orientation of the HMD; and

detecting the gesture input corresponding to the detection of the end of the virtual session when it is determined, based on the position and orientation of the HMD, that the HMD remains stationary for greater than or equal to a set amount of time.

5. The method of claim 1 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting the position of the HMD; and

detecting the gesture input corresponding to the detection of the end of the virtual session when, based on the position of the HMD, it is determined that the HMD is positioned at less than or equal to a preset distance of a computing device in the ambient environment, the computing device being operably coupled to the HMD, and external to the HMD.

6. The method of claim 1 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting the position of the HMD;

detecting a movement path of the HMD based on the position of the HMD; and

detecting the gesture input corresponding to the detection of the end of the virtual session when, based on the movement path of the HMD, it is determined that the movement path of the HMD is aligned with a computing device in the ambient environment, the computing device being operably coupled to the HMD, and external to the HMD.

7. The method of claim 6 , detecting a movement path of the HMD based on the position of the HMD including:

detecting a movement direction of the HMD;

detecting a movement velocity of the HMD; and

detecting the gesture input corresponding to the end of the virtual session when the movement velocity of the HMD in the detected movement direction is greater than or equal to a set velocity.

8. The method of claim 1 , initiating processing related to the at least one user input received during the virtual session including:

automatically initiating processing at a computing device in the ambient environment, operably coupled to the HMD, and external to the HMD without user input.

9. A system, comprising:

a computing device configured to generate a virtual reality environment, the computing device including:

a memory storing executable instructions; and

a processor configured to execute the instructions to cause the computing device to:

generate a virtual environment in a head mounted display (HMD) device operating in an ambient environment;

initiate a virtual session and receiving at least one user input for executing at least one corresponding command in the virtual environment while the virtual session is in an active state;

detect at least one of a position or an orientation of the HMD in the ambient environment;

detect a gesture input based on the at least one of the position or the orientation of the HMD;

detect an end of the virtual session based on the detection of the gesture input; and

initiate processing related to the at least one user input received during the virtual session while in the active state in response to the detected end of the virtual session.

10. The system of claim 9 , wherein, in detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment, the instructions cause the computing device to:

detect an orientation of the HMD;

compare the detected orientation of the HMD to a set orientation; and

detect the gesture input corresponding to the detection of the end of the virtual session when, based on the comparison it is determined that the detected orientation of the HMD matches the set orientation of the HMD for greater than or equal to a set amount of time.

11. The system of claim 10 , wherein the set orientation of the HMD corresponds to a horizontal alignment of a centerline of the HMD worn by the user.

12. The system of claim 9 , wherein, in detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment, the instructions cause the computing device to:

detect the position and the orientation of the HMD; and

detect the gesture input corresponding to the detection of the end of the virtual session when it is determined, based on the position and orientation of the HMD, that the HMD remains stationary for greater than or equal to a set amount of time.

13. The system of claim 9 , wherein, in detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment, the instructions cause the computing device to:

detect the position of the HMD; and

detect the gesture input corresponding to the detection of the end of the virtual session when, based on the position of the HMD, it is determined that the HMD is positioned at less than or equal to a set distance of a computing device in the ambient environment, the computing device being operably coupled to the HMD, and external to the HMD.

14. The system of claim 9 , wherein, in detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment, the instructions cause the computing device to:

detect the position of the HMD;

detect a movement path of the HMD based on the position of the HMD; and

detect the gesture input corresponding to the detection of the end of the virtual session when, based on the detected movement path of the HMD, it is determined that the detected movement path of the HMD is aligned with a computing device in the ambient environment, the computing device being operably coupled to the HMD, and external to the HMD.

15. The method of claim 14 , wherein, in detecting a movement path of the HMD based on the position of the HMD, the instructions cause the computing device to:

detect a movement direction of the HMD;

detect a movement velocity of the HMD; and

detect the gesture input corresponding to the end of the virtual session when the movement velocity of the HMD in the detected movement direction is greater than or equal to a set velocity.

16. The system of claim 9 , wherein, in initiating processing related to the at least one user input received during the virtual session, the instructions cause the computing device to:

automatically initiate processing at a computing device in the ambient environment, operably coupled to the HMD, and external to the HMD without user input.

17. A computer program product embodied on a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method, the method comprising:

generating a virtual environment in a head mounted display (HMD) device operating in an ambient environment;

initiating a virtual session and receiving at least one user input for executing at least one corresponding command in the virtual environment while the virtual session is in an active state;

detecting a position and an orientation of the HMD in the ambient environment;

detecting a gesture input based on the at least one of the position or the orientation of the HMD;

detecting an end of the virtual session based on the detection of the gesture input; and

automatically initiating, in response to the detected end of the virtual session, processing related to the at least one user input received during the virtual session while in the active state, at a computing device in the ambient environment, the computing device being operably coupled to the HMD and external to the HMD.

18. The computer program product of claim 17 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting an orientation of the HMD;

comparing the detected orientation of the HMD to a set orientation; and

detecting the gesture input corresponding to the detection of the end of the virtual session when, based on the detected orientation of the HMD, it is determined that the HMD is oriented substantially horizontally for greater than or equal to a set amount of time.

19. The computer program product of claim 17 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting the position and the orientation of the HMD; and

detecting the gesture input corresponding to the detection of the end of the virtual session when it is determined, based on the position and orientation of the HMD, that the HMD remains stationary for greater than or equal to a set amount of time.

20. The computer program product of claim 17 , detecting a gesture input based on the at least one of the position or the orientation of the HMD in the ambient environment including:

detecting the position of the HMD; and

detecting the gesture input corresponding to the detection of the end of the virtual session when, based on the detected and tracked position of the HMD, it is determined that the HMD is positioned at less than or equal to a set distance of the computing device.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2016
From: TOFF, JASON
To: GOOGLE INC.
Reel/Frame 040595/0297 →
Continuity (1)
Related Publication 20180157344A1 · Jun 7, 2018
Cited By (1)
US 12,664,696