IP Library Granted Patent US 10,976,830
Granted Patent B2
US 10,976,830 · App. 16/437,888 · Granted Apr 13, 2021

Unified virtual reality platform

Inventors: Vishal Sharma (Los Altos, CA); Elhum Amjadi (Foster City, CA)
G06F3/017G06F3/011G06F3/167G06F9/541G06Q30/0641G06Q50/16H04L67/10H04L67/38G06F2203/0381G06Q30/0601
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,976,830
App. No.
16/437,888
Granted
Apr 13, 2021
Kind
B2
Abstract

Techniques for providing a unified computational platform for implementing virtual reality systems. In an aspect, a virtual reality (VR) client includes input and output modules for generating a VR user experience. The client may also incorporate an assistant interface that couples the input and output modules to an assistant server using a semantic information interchange language (SIIL). The assistant server may include a command inference block for converting user-input SIIL messages to command-input SIIL messages, thereby encoding user VR inputs into a general command set that may be recognized and processed by both VR and non-VR modules. Further aspects disclose techniques for seamless integration between virtual environments and joint VR sessions supporting multiple users.

Claims (57)

1. An apparatus comprising:

a command inference block configured to receive from a client a user-input message according to a first protocol specifying an input to a virtual reality system, and to convert the user-input message into a command-input message according to the first protocol; and

at least one plugin module configured to:

transmit a command input specified by the command-input message to an application module;

translate a custom service message received from the application module into a first service message according to the first protocol; and

transmit the first service message to the client.

2. The apparatus of claim 1 , the user-input message specifying at least one of a gesture performed by the user and a voice input.

3. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message; and

encapsulate the proposed command in the command-input SIIL message.

4. The apparatus of claim 1 , the context comprising at least one of co-presence of other individuals in a joint VR environment when one user generates an input, the user's geographic location when generating the input, a time of day when the user input is made, and a direction in which a physical input such as a gesture is made.

5. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message;

if a rejection of the proposed command is received from the user, encapsulate another command suggested by the user in the command-input SILL message; and

update the library based on the another command suggested by the user.

6. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message;

if the library does not contain a proposed command based on the received context and the user-input message, encapsulate another command suggested by a second user in the command-input SIIL message; and

update the library based on the another command suggested by the second user.

7. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a plurality of user-input messages from a plurality of users;

receive a context of each user-input message;

retrieve a proposed command from a library based on the plurality of received contexts and the plurality of user-input messages; and

encapsulate the proposed command in the command-input SIIL message.

8. The apparatus of claim 1 , the at least one plugin module comprising a first virtual reality (VR) plugin and a second non-virtual reality (non-VR) plugin, the first VR plugin configured to exchange at least one parameter of the user with the second non-VR plugin using a message according to the first protocol, the at least one parameter comprising a credential or authorization associated with a payment instrument owned by the user, an assigned status of the user, and a user profile.

9. The apparatus of claim 1 , the at least one plugin module comprising a first virtual reality (VR) plugin and a second virtual reality (VR) plugin, the first VR plugin configured to exchange with the second VR plugin using messages according to the first protocol a virtual environment parameter comprising at least one of a virtual environment coordinate system, relative position of a user in the virtual environment coordinate system, and visual data associated with the virtual environment.

10. The apparatus of claim 9 , the first VR plugin configured to exchange data with the second VR plugin using a network communications channel.

11. The apparatus of claim 1 , the first service message comprising an instruction to the client to change an output signal generated by a virtual reality output module.

12. The apparatus of claim 1 , further comprising a server computer for implementing the command inference block and the at least one plugin module, the server computer configured to communicate with the client using a network connection.

13. The apparatus of claim 1 , the command inference block further configured to receive a second user-input message according to the first protocol, and to convert the combination of the user-input message and the second user-input message into a single command-input message according to the first protocol.

14. A method for converting a user-input digital message to a command-input digital message for a virtual reality system, the method comprising:

receiving at least one user-input message according to a first protocol;

receiving a context for each of the at least one user-input message;

retrieving a proposed command, if available, from a digital library mapping user input and context to command inputs;

if a user rejection of the proposed command is not received, encapsulating the proposed command in at least one command-input message according to the first protocol; and

transmitting the command-input message to an application module.

15. The method of claim 14 , further comprising:

if a user rejection of the proposed command is received, encapsulating an alternative command specified by a user in the at least one command-input message according to the first protocol; and

updating the digital library to map the user input and context to the alternative command specified by the user.

16. The method of claim 14 , further comprising:

if the proposed command is not available from the digital library mapping user input and context to command inputs, receiving an alternative command specified by an alternative user; and

encapsulating the alternative command in the at least one command-input message according to the first protocol.

17. The method of claim 16 , the receiving at least one user-input message according to a first protocol comprising receiving at least one user-input message from each of a plurality of users, the retrieving the proposed command comprising retrieving a single proposed command for the plurality of user-input messages from the plurality of users.

18. A computing device including a memory holding instructions executable by a processor to:

receive from a client a user-input message according to a first protocol specifying an input to a virtual reality system;

convert the user-input message into a command-input message according to the first protocol;

transmit a command input specified by the command-input message to an application module;

translate a custom service message received from the application module into a first service message according to the first protocol; and

transmit the first service message to the client.

19. The device of claim 18 , the user-input message specifying at least one of a gesture performed by the user and a voice input.

20. The device of claim 18 , the memory further holding instructions executable by a processor to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message; and

encapsulate the proposed command in the command-input SIIL message.

Continuity (3)
Continuation 15279215 · Sep 28, 2016
Provisional Application 62233601 · Sep 28, 2015
Related Publication 20190294255A1 · Sep 26, 2019