IP Library Granted Patent US 10,698,561
Granted Patent B2
US 10,698,561 · App. 15/620,326 · Granted Jun 30, 2020

Intelligent command batching in an augmented and/or virtual reality environment

Inventor: Ian MacGillivray (New York, NY)
Assignee: GOOGLE LLC
G06F3/04815G06F3/011G06F9/46G06T11/001G06T15/20G06F3/012G06F3/013G06F3/017G06F3/0346G06T19/006G06T2219/024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,698,561
App. No.
15/620,326
Granted
Jun 30, 2020
Kind
B2
Abstract

In an augmented reality and/or a virtual reality system, detected commands may be intelligently batched to preserve the relative order of the batched commands while maintaining a fluid virtual experience for the user. Commands detected in the virtual environment may be assigned to a batch command, of a plurality of batch commands, based on a temporal window in which the command(s) are detected, based on an operational type associated with the command(s), or based on a spatial position at which the command is detected in the virtual environment. The commands included in a batched set of commands may be executed in response to an un-do command and/or a re-do command and/or a re-play command.

Claims (104)

1. A method, comprising:

detecting, in a three dimensional (3D) virtual environment generated by a computing device, a plurality of commands to be executed in the virtual environment, wherein at least one of the detected commands is a 3D input detected in a spatial volume of the 3D virtual environment;

executing the plurality of commands;

assigning each command of the plurality of commands to a batch, of a plurality of batches, including, for each command of the plurality of commands:

in a first mode, assigning the command to the batch based on a moving temporal window associated with the detection of the command,

in a second mode, assigning the command to the batch based on a classification of an operation to be executed in connection with the command, and

in a third mode, assigning the command to the batch based on a spatial position in the 3D virtual environment associated with the detection of the command;

executing a supplementary command associated with at least one of the plurality of batches of commands; and

executing the supplementary command.

2. The method of claim 1 , wherein executing the supplementary command includes:

detecting an un-do command;

identifying an un-do batch, of the plurality of batches, associated with the un-do command; and

reversing a previous execution of commands associated with the identified un-do batch.

3. The method of claim 2 , wherein executing the supplementary command includes:

detecting a re-do command;

identifying a re-do batch, of the plurality of batches, associated with the re-do command; and

executing commands associated with the identified re-do batch.

4. The method of claim 3 , wherein executing commands associated with the identified re-do batch includes re-executing a batch of previously reversed commands associated with an un-do batch of commands.

5. The method of claim 1 , wherein executing the supplementary command includes:

detecting a re-play command;

identifying a range of commands, of the plurality of commands, included in the re-play command;

identifying a beginning command corresponding to a first command of the range of commands to be re-played;

identifying an end command corresponding to a last command of the range of commands to be re-played; and

sequentially executing the range of commands, from the first command to the last command.

6. The method of claim 5 , wherein the re-play command includes one or more batches of the plurality of batches of commands.

7. The method of claim 5 , wherein sequentially executing the range of commands includes re-executing one or more batches, of the plurality of batches of commands, of previously reversed commands associated with one or more un-do batches of commands.

8. The method of claim 1 , wherein the assigning of the command to the batch in the first mode includes assigning each command of the plurality of commands to the batch of the plurality of batches based on the moving temporal window associated with the detection of the command, including:

monitoring time associated with interaction in the virtual environment;

assigning commands detected within a set window of time to a current temporal window;

re-setting the current temporal window at set time intervals; and

updating the commands assigned to the current temporal window based on the re-set temporal window.

9. The method of claim 1 , wherein the assigning of the command to the batch in the second mode includes assigning each command of the plurality of commands to the batch of the plurality of batches based on the classification of an operation to be executed in connection with the command, including:

detecting an operational category, of a plurality of operational categories, associated with each of the detected commands;

assigning each of the detected commands to a batch corresponding to the detected operational category; and

identifying a batch, of the plurality of batches of commands for execution.

10. The method of claim 9 , wherein executing the supplementary command includes:

detecting the supplementary command;

identifying the operational category associated with the detected supplementary command; and

executing the supplementary command including the batch of commands assigned to the corresponding operational category.

11. The method of claim 1 , wherein the assigning of the command to the batch in the third mode includes assigning each command of the plurality of commands to the batch of the plurality of batches based on the spatial position in the 3D virtual environment associated with the detection of the command, including:

setting a plurality of spatial areas in the virtual environment, each of the plurality of spatial areas being defined by respective set of spatial boundaries; and

assigning each of the detected commands to a corresponding spatial area based on a detection position of the command within the corresponding spatial area.

12. The method of claim 11 , wherein executing the supplementary command includes:

detecting the supplementary command;

identifying the spatial area associated with the detected supplementary command; and

executing the supplementary command including the batch of commands assigned to the corresponding spatial area.

13. The method of claim 1 , wherein detecting the plurality of commands includes:

detecting a first plurality of commands issued by a first user, wherein the first plurality of commands are directed for execution with respect to a virtual object in the virtual environment;

assigning the first plurality of commands to a first batch;

detecting a second plurality of commands issued by a second user, wherein the second plurality of commands are directed for execution with respect to the virtual object in the virtual environment; and

assigning the second plurality of commands to a second batch.

14. The method of claim 13 , wherein

detecting the first plurality of commands issued by the first user includes detecting the first plurality of commands in a first spatial area of the virtual environment, and

detecting the second plurality of commands issued by the second user includes detecting the second plurality of commands in a second spatial area of the virtual environment.

15. The method of claim 14 , wherein executing the supplementary command includes:

detecting the supplementary command in the first spatial area of the virtual environment or in or the second spatial area of the virtual environment;

executing the supplementary command in accordance with the first batch in response to detection of the supplementary command in the first spatial area; and

executing the supplementary command in accordance with the second batch in response to detection of the supplementary command in the second spatial area.

16. A computer program product comprising a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to execute a method, the method comprising:

detecting, in a three dimensional (3D) virtual environment generated by a computing device, a plurality of commands to be executed in the virtual environment, wherein at least one of the detected commands is a 3D input detected in a spatial volume of the 3D virtual environment;

executing the plurality of commands;

assigning each command of the plurality of commands to a batch, of a plurality of batches, including, for each command of the plurality of commands:

in a first mode, assigning the command to the batch based on a moving temporal window associated with the detection of the command,

in a second mode, assigning the command to the batch based on a classification of an operation to be executed in connection with the command, and

in a third mode, assigning the command to the batch based on a spatial position in the 3D virtual environment associated with the detection of the command; and

executing a supplementary command associated with at least one of the plurality of batches of commands.

17. A computer program product comprising a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to execute a method, the method comprising:

generating a three dimensional (3D) virtual environment;

detecting a plurality of commands to be executed in the virtual environment, wherein at least one of the detected commands is a 3D input detected in a spatial volume of the 3D virtual environment;

assigning each command of the plurality of commands to a batch, of a plurality of batches, including, for each command of the plurality of commands:

in a first mode, assigning the command to the batch based on a moving temporal window associated with the detection of the command,

in a second mode, assigning the command to the batch based on a classification of an operation to be executed in connection with the command, and

in a third mode, assigning the command to the batch based on a spatial position in the 3D virtual environment associated with the detection of the command;

detecting a supplementary command associated with at least one of the plurality of batches of commands; and

executing the supplementary command.

18. The computer program product of claim 17 , wherein detecting the supplementary command and executing the supplementary command includes at least one of:

detecting and executing an un-do command, including:

identifying an un-do batch, of the plurality of batches, associated with the un-do command; and

reversing a previous execution of commands associated with the identified un-do batch;

detecting and executing a re-do command, including:

identifying a re-do batch, of the plurality of batches, associated with the re-do command; and

executing commands associated with the identified re-do batch, including re-execution one or more previously undone commands; or

detecting and executing a re-play command, including:

identifying a range of commands, of the plurality of commands, included in the re-play command, the range of commands including one or more batches, of the plurality of batches; and

sequentially executing the range of commands, including re-executing one or more batches, of the plurality of batches of commands, of previously reversed commands associated with one or more un-do batches of commands.

19. The computer program product of claim 17 , wherein the assigning of the command to the batch in the first mode includes assigning of each command of the plurality of commands to the batch of the plurality of batches based on the moving temporal window associated with the detection of the command, including:

monitoring time associated with interaction in the virtual environment;

assigning commands detected within a set window of time to a current temporal window;

re-setting the current temporal window at set time intervals; and

updating the commands assigned to the current temporal window based on the re-set temporal window.

20. The computer program product of claim 17 , wherein the assigning of the command to the batch in the second mode includes assigning each command of the plurality of commands to the batch of the plurality of batches based on the classification of an operation to be executed in connection with the command, including:

detecting an operational category, of a plurality of operational categories, associated with each of the detected commands;

assigning each of the detected commands to a batch corresponding to the detected operational category; and

identifying a batch, of the plurality of batches of commands for execution; and

the executing of the supplementary command includes:

identifying the operational category associated with the detected supplementary command; and

executing the supplementary command including the batch of commands assigned to the corresponding operational category.

21. The computer program product of claim 17 ,

wherein the assigning of the command to the batch in the third mode includes assigning each command of the plurality of commands to the batch of the plurality of batches based on the spatial position in the 3D virtual environment associated with the detection of the command, including:

setting a plurality of spatial areas in the virtual environment, each of the plurality of spatial areas being defined by respective set of spatial boundaries; and

assigning each of the detected commands to a spatial area based on a detection position of the command within a corresponding spatial area; and

the executing of the supplementary command includes:

identifying the spatial area associated with the detected supplementary command; and

executing the supplementary command including the batch of commands assigned to the corresponding spatial area.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2017
From: MACGILLIVRAY, IAN
To: GOOGLE INC.
Reel/Frame 043379/0013 →
Continuity (1)
Related Publication 20180356956A1 · Dec 13, 2018