IP Library Granted Patent US 10,841,534
Granted Patent B2
US 10,841,534 · App. 15/951,685 · Granted Nov 17, 2020

Real-world awareness for virtual reality

Inventor: Jonathan S. Kaufthal (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N7/15G06K9/00268H04M3/42042
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,841,534
App. No.
15/951,685
Granted
Nov 17, 2020
Kind
B2
Abstract

Techniques and systems for providing real-world awareness for virtual reality are disclosed. In response to determining that a virtual reality application is in use, an awareness module can begin monitoring for external signals by at least accessing an available sensor of a computing device. In response to detecting the at least one external signal, the awareness module can determine a level of notification for an interruption notification of the virtual reality application based on the at least one external signal. The awareness module can receive context of the virtual reality application. A type for the interruption notification of the virtual reality application can be determined based on the determined level of notification and the context of the virtual reality application. The interruption notification may be provided to the virtual reality application based on the determined type.

Claims (74)

1. A method comprising:

determining that a virtual reality application is in use;

in response to determining that the virtual reality application is in use, monitoring for external signals by at least accessing an available sensor of a computing device;

detecting at least one external signal while monitoring for the external signals;

in response to detecting the at least one external signal, determining a level of notification for an interruption notification of the virtual reality application based on the at least one external signal;

receiving context of the virtual reality application;

after determining the level of notification and receiving the context, determining a type for the interruption notification of the virtual reality application based on both the determined level of notification and the context of the virtual reality application, wherein the type for the interruption notification comprises a degree of transparency of the interruption notification; and

providing the interruption notification to the virtual reality application based on the determined type.

2. The method of claim 1 , wherein the at least one external signal comprises at least one of an audio signal or a video signal.

3. The method of claim 1 , wherein the determining of the type for the interruption notification of the virtual reality application comprises:

accessing a mapping resource storing a mapping data structure to identify the type associated with the determined level of notification and the context of the virtual reality application.

4. The method of claim 1 , wherein the receiving of the context of the virtual reality application comprises:

detecting user input signals to the virtual reality application or accessing a graph structure.

5. The method of claim 1 , wherein the determining of the level of notification for the interruption notification of the virtual reality application comprises:

from the at least one external signal, applying a signal analysis;

assigning a notification value to a result of the signal analysis; and

for each result with a notification value that meets a threshold value, mapping the notification value to an appropriate level of notification, the appropriate level being the level of notification for the interruption notification.

6. The method of claim 5 , wherein the at least one external signal is a video signal and applying the signal analysis comprises:

performing facial recognition on the video signal to identify one or more people in the video signal;

obtaining user context information, wherein the user context information comprises contact information of the user of the virtual reality application; and

determining whether each of the one or more identified people is a person of interest to the user based on the obtained contact information; and

wherein assigning the notification value to the result of the signal analysis comprises:

assigning the notification value based on whether each of the one or more people are determined to be the person of interest and a relationship to the user.

7. The method of claim 5 , wherein the at least one external signal is an audio signal and applying the signal analysis comprises:

performing voice recognition on the audio signal to identify one or more people in the audio signal;

obtaining user context information, wherein the user context information comprises user contact information of the user of the virtual reality application; and

determining whether each of the one or more identified people is a person of interest to the user based on the obtained contact information; and

wherein assigning the notification value to the result of the signal analysis comprises:

assigning the notification value based on whether each of the one or more people are determined to be the person of interest and a relationship to the user.

8. The method of claim 5 , wherein the at least one external signal is a video signal and applying the signal analysis comprises performing a movement analyses on the video signal to identify a movement level in the video signal and wherein assigning the notification value to the result of the signal analysis comprises assigning the notification value based on the identified movement level.

9. The method of claim 5 , wherein the at least one external signal is an audio signal and applying the signal analysis comprises performing a sound analyses on the audio signal to identify a sound level in the audio signal and wherein assigning the notification value to the result of the signal analysis comprises assigning the notification value based on the identified sound level.

10. A system comprising:

a processing system;

a storage system; and

instructions for supporting real-world awareness stored on the storage system that, when executed by the processing system, direct the processing system to:

determine that a virtual reality application is in use;

in response to determining that the virtual reality application is in use, begin monitoring for external signals by at least accessing an available sensor of a computing device;

detect at least one external signal while monitoring for the external signals;

in response to detecting the at least one external signal, determine a level of notification for an interruption notification of the virtual reality application based on the at least one external signal;

receive context of the virtual reality application;

after determining the level of notification and receiving the context, determine a type for the interruption notification of the virtual reality application based on both the determined level of notification and the context of the virtual reality application, wherein the type for the interruption notification comprises a degree of transparency of the notification interruption; and

provide the interruption notification to the virtual reality application based on the determined type.

11. The system of claim 10 , wherein the at least one external signal comprises at least one of an audio signal or a video signal.

12. The system of claim 10 , wherein the instructions that determine the type for the interruption notification of the virtual reality application further direct the processing system to:

access a mapping resource storing a mapping data structure to identify the type associated with the determined level of notification and the context of the virtual reality application.

13. The system of claim 10 , wherein the instructions that receive of the context of the virtual reality application further direct the processing system to:

detect user input signals to the virtual reality application or accessing a graph structure.

14. The system of claim 10 , wherein the instructions that determine of the level of notification for the interruption notification of the virtual reality application further direct the processing system to:

from the at least one external signal, apply a signal analysis;

assign a notification value to a result of the signal analysis; and

for each result with a notification value that meets a threshold value, map the notification value to an appropriate level of notification, the appropriate level being the level of notification for the interruption notification.

15. One or more computer readable storage media having instructions stored thereon that, when executed by a processing system, direct the processing system to at least:

determine that a virtual reality application is in use;

in response to determining that the virtual reality application is in use, begin monitoring for external signals by at least accessing an available sensor of a computing device;

detect at least one external signal while monitoring for the external signals;

in response to detecting the at least one external signal, determine a level of notification for an interruption notification of the virtual reality application based on the at least one external signal;

receive context of the virtual reality application;

after determining the level of notification and receiving the context, determine a type for the interruption notification of the virtual reality application based on both the determined level of notification and the context of the virtual reality application, wherein the type for the interruption notification comprises a degree of transparency of the notification interruption; and

provide the interruption notification to the virtual reality application based on the determined type.

16. The media of claim 15 , wherein the at least one external signal comprises at least one of an audio signal or a video signal.

17. The media of claim 15 , wherein the instructions that determine the type for the interruption notification of the virtual reality application further direct the processing system to at least:

access a mapping resource storing a mapping data structure to identify the type associated with the determined level of notification and the context of the virtual reality application.

18. The media of claim 15 , wherein the instructions that receive of the context of the virtual reality application further direct the processing system to at least:

detect user input signals to the virtual reality application or accessing a graph structure.

19. The media of claim 15 , wherein the instructions that determine of the level of notification for the interruption notification of the virtual reality application further direct the processing system to at least:

from the at least one external signal, apply a signal analysis;

assign a notification value to a result of the signal analysis; and

for each result with a notification value that meets a threshold value, map the notification value to an appropriate level of notification, the appropriate level being the level of notification for the interruption notification.

20. The media of claim 19 , wherein the at least one external signal is a video signal and the instructions that apply the signal analysis further direct the processing system to at least:

perform facial recognition on the video signal to identify one or more people in the video signal;

obtain user context information, wherein the user context information comprises contact information of the user of the virtual reality application; and

determine whether each of the one or more identified people is a person of interest to the user based on the obtained contact information; and

wherein the instructions that assign the notification value to the result of the signal analysis further direct the processing system to at least:

assign the notification value based on whether each of the one or more people are determined to be the person of interest and a relationship to the user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2018
From: KAUFTHAL, JONATHAN S.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 045523/0517 →
Continuity (1)
Related Publication 20190320138A1 · Oct 17, 2019