IP Library Granted Patent US 10,353,474
Granted Patent B2
US 10,353,474 · App. 15/279,215 · Granted Jul 16, 2019

Unified virtual reality platform

Inventors: Vishal Sharma (Los Altos, CA); Elhum Amjadi (Foster City, CA)
Assignee: Microsoft Technology Licensing, LLC
G06F3/017G06F3/011G06F3/167G06F9/541H04L67/10H04L67/38G06F2203/0381G06Q30/0601
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,353,474
App. No.
15/279,215
Granted
Jul 16, 2019
Kind
B2
Abstract

Techniques for providing a unified computational platform for implementing virtual reality systems. In an aspect, a virtual reality (VR) client includes input and output modules for generating a VR user experience. The client also incorporates an assistant interface that couples the input and output modules to an assistant server using a semantic information interchange language (SIIL). The assistant server includes a command inference block for converting user-input SIIL messages to command-input SIIL messages, thereby encoding user VR inputs into a general command set that is recognized and processed by both VR and non-VR modules. Further aspects include techniques for seamless integration between virtual environments and joint VR sessions supporting multiple users.

Claims (56)

1. A server apparatus comprising a central processing unit (CPU), and a storage unit storing instructions executable by the CPU, the instructions configured to implement:

a command inference block configured to, using a communicator, receive from a client a user-input message according to a universal protocol specifying an input to a virtual reality system, and to convert the user-input message into a command-input message according to the universal protocol; and

at least one plugin module configured to:

transmit using a communicator a command input specified by the command-input message to an application module;

translate a custom service message received from the application module into a universal service message according to the universal protocol; and

transmit using a communicator the universal service message to the client.

2. The apparatus of claim 1 , the user-input message specifying at least one of a gesture performed by the user and a voice input.

3. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message; and

encapsulate the proposed command in the command-input message.

4. The apparatus of claim 1 , the context comprising at least one of co-presence of other individuals in a joint VR environment when one user generates an input, the user's geographic location when generating the input, a time of day when the user input is made, and a direction in which a physical input such as a gesture is made.

5. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message;

if a rejection of the proposed command is received from the user, encapsulate another command suggested by the user in the command-input message; and

update the library based on the another command suggested by the user.

6. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message;

if the library does not contain a proposed command based on the received context and the user-input message, encapsulate another command suggested by a second user in the command-input message; and

update the library based on the another command suggested by the second user.

7. The apparatus of claim 1 , the command inference block configured to convert the user-input message into the command-input language by being configured to:

receive a plurality of user-input messages from a plurality of users; receive a context of each user-input message;

retrieve a proposed command from a library based on the plurality of received contexts and the plurality of user-input messages; and

encapsulate the proposed command in the command-input message.

8. The apparatus of claim 1 , the at least one plugin module comprising a first virtual reality (VR) plugin and a second non-virtual reality (non-VR) plugin, the first VR plugin configured to exchange at least one parameter of the user with the second non-VR plugin using a message according to the universal protocol, the at least one parameter comprising a credential or authorization associated with a payment instrument owned by the user, an assigned status of the user, and a user profile.

9. The apparatus of claim 1 , the at least one plugin module comprising a first virtual reality (VR) plugin and a second virtual reality (VR) plugin, the first VR plugin configured to exchange with the second VR plugin using messages according to the universal protocol a virtual environment parameter comprising at least one of a virtual environment coordinate system, relative position of a user in the virtual environment coordinate system, and visual data associated with the virtual environment.

10. The apparatus of claim 9 , the first VR plugin configured to exchange data with the second VR plugin using a network communications channel.

11. The apparatus of claim 1 , the universal service message comprising an instruction to the client to change an output signal generated by a virtual reality output module.

12. The apparatus of claim 1 , further comprising a server computer for implementing the command inference block and the at least one plugin module, the server computer configured to communicate with the client using a network connection.

13. The apparatus of claim 1 , the command inference block further configured to receive a second user-input message according to the universal protocol, and to convert the combination of the user-input message and the second user-input message into a single command-input message according to the universal protocol.

14. A method for converting a user-input digital message to a command-input digital message for a virtual reality system comprising a communicator, a central processing unit (CPU), and a storage unit storing instructions executable by the CPU, the method comprising:

receiving using the communicator at least one user-input message according to a universal protocol;

receiving a context for each of the at least one user-input message;

retrieving a proposed command, if available, from a digital library mapping user input and context to command inputs;

if a user rejection of the proposed command is not received, encapsulating the proposed command in at least one command-input message according to the universal protocol; and

transmitting using the communicator the command-input message to an application module.

15. The method of claim 14 , further comprising:

if a user rejection of the proposed command is received, encapsulating an alternative command specified by a user in the at least one command-input message according to the universal protocol; and

updating the digital library to map the user input and context to the alternative command specified by the user.

16. The method of claim 14 , further comprising:

if the proposed command is not available from the digital library mapping user input and context to command inputs, receiving an alternative command specified by an alternative user; and

encapsulating the alternative command in the at least one command-input message according to the universal protocol.

17. The method of claim 16 , the receiving at least one user-input message according to a universal protocol comprising receiving at least one user-input message from each of a plurality of users, the retrieving the proposed command comprising retrieving a single proposed command for the plurality of user-input messages from the plurality of users.

18. A computing device including a communicator, a processor, and a memory holding instructions executable by the processor to:

receive from a client using the communicator a user-input message according to a universal protocol specifying an input to a virtual reality system;

convert the user-input message into a command-input message according to the universal protocol;

transmit using the communicator a command input specified by the command-input message to an application module;

translate a custom service message received from the application module into a universal service message according to the universal protocol; and

transmit using the communicator the universal service message to the client.

19. The device of claim 18 , the user-input message specifying at least one of a gesture performed by the user and a voice input.

20. The device of claim 18 , the memory further holding instructions executable by the processor to:

receive a context of the user-input message;

retrieve a proposed command from a library based on the received context and the user-input message; and

encapsulate the proposed command in the command-input message.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2017
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 044402/0248 →
MERGER Recorded Dec 8, 2017
From: WAND LABS, INC.
To: MICROSOFT CORPORATION
Reel/Frame 044338/0698 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2016
From: SHARMA, VISHAL; AMJADI, ELHUM
To: WAND LABS, INC.
Reel/Frame 039881/0166 →
Continuity (2)
Provisional Application 62233601 · Sep 28, 2015
Related Publication 20170090589A1 · Mar 30, 2017
Cited By (1)
US 12,646,518