IP Library Patent Application 15159743
Patent Application
App. No. 15/159,743

VIRTUAL PERSONIFICATION FOR AUGMENTED REALITY SYSTEM

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
15/159,743
Abstract

A head mounted device (HMD) includes a transparent display, a first set of sensors, a second set of sensors, and a processor. The first set of sensors measures first sensor data including an identification of a user of the HMD and a biometric state of the user of the HMD. The second set of sensors measures second sensor data including a location of the HMD and ambient metrics based on the location of the HMD. The HMD determines a user-based context based on the first sensor data, determines an ambient-based context based on the second sensor data, determines an application context within an AR application implemented by the processor, identifies a virtual fictional character based on a combination of the user-based context, the ambient-based context, and the application context, and displays the virtual fictional character in the transparent display.

Claims (85)

1 . A head mounted device (HMD) comprising:

a transparent display;

a first set of sensors configured to measure first sensor data including an identification of a user of the HMD and a biometric state of the user of the HMD;

a second set of sensors configured to measure second sensor data including a location of the HMD and ambient metrics based on the location of the HMD; and

a processor configured to perform operations comprising:

determine a user-based context based on the first sensor data,

determine an ambient-based context based on the second sensor data,

determine an application context within an AR application implemented by the processor,

identify a virtual fictional character based on a combination of the user-based context, the ambient-based context, and the application context, and

display the virtual fictional character in the transparent display.

2 . The HMD of claim 1 , wherein the processor is further configured to:

identify an object depicted in an image generated by a camera of the HMD, the object being located in a line of sight of the user through the transparent display;

access the virtual character based on an identification of the object; and

adjust a size and a position of the virtual character in the transparent display based on a relative position between the object and the camera.

3 . The HMD of claim 1 , wherein the first sensor data includes at least one of a heart rate, a blood pressure, or brain activity,

wherein the second sensor data includes at least one of an orientation and position of the HMD, an ambient pressure, an ambient humidity level, or an ambient light level, and

wherein the processor is further configured to identify a task performed by the user.

4 . The HMD of claim 1 , wherein the processor is further configured to:

identify a character content for the virtual fictional character, the character content based on a combination of the user-based context, the ambient-based context, and the application context, the character content comprising an animation content and a speech content.

5 . The HMD of claim 4 , wherein the processor is further configured to:

detect a change in at least one of the user-based context, the ambient-based context, and the application context; and

adjust the character content based on the change.

6 . The HMD of claim 1 , wherein the processor is further configured to:

identify the virtual fictional character based on the application context:

record an input from the user of the HMD;

communicate the input to a remote server;

receive audio data in response to the input; and

animate the virtual fictional character based on the audio data.

7 . The HMD of claim 1 , wherein the processor is further configured to:

identify the virtual fictional character based on a task performed by the user; and

generate a character content for the virtual fictional character, the character content based on a combination of the task, the user-based context, the ambient-based context, and the application context, the character content comprising an animation content and a speech content.

8 . The HMD of claim 1 , wherein the processor is further configured to:

compare the first sensor data with first reference sensor data for a task performed by the user;

determine the user-based context based on the comparison of the first sensor data with the first reference sensor data;

compare the second sensor data with second reference sensor data for the task performed by the user; and

determine the ambient-based context based on the comparison of the second sensor data with the second reference sensor data.

9 . The HMD of claim 8 , wherein the first reference sensor data includes a set of physiological data ranges for the user corresponding to the first set of sensors, a first set of the physiological data ranges corresponding to a first virtual character, and a second set of physiological data ranges corresponding to a second virtual character,

wherein the second reference sensor data includes a set of ambient data ranges for the HMD corresponding to the second set of sensors, a first set of ambient data ranges corresponding to the first virtual character, and a second set of ambient data ranges corresponding to the second virtual character.

10 . The HMD of claim 9 , wherein the processor is further configured to:

change the virtual character based on whether the first sensor data transgress the set of physiological data ranges for the user, and whether the second sensor data transgress the set of ambient data ranges for the HMD.

11 . A method comprising:

measuring first sensor data including an identification of a user of a head mounted device (HMD) and a biometric state of the user of the HMD with a first set of sensors of the HMD;

measuring second sensor data including a location of the HMD and ambient metrics based on the location of the HMD with a second set of sensors;

determining a user-based context based on the first sensor data;

determining an ambient-based context based on the second sensor data;

determining, using a processor of the HMD, an application context within an AR application implemented by the processor of the HMD;

identifying a virtual fictional character based on a combination of the user-based context, the ambient-based context, and the application context; and

displaying the virtual fictional character in a transparent display of the HMD.

12 . The method of claim 11 , further comprising:

identifying an object depicted in an image generated by a camera of the HMD, the object being located in a line of sight of the user through the transparent display;

accessing the virtual fictional character based on an identification of the object; and

adjusting a size and a position of the virtual character in the transparent display based on a relative position between the object and the camera.

13 . The method of claim 11 , further comprising:

identifying a task performed by the user,

wherein the first sensor data includes at least one of a heart rate, a blood pressure, or brain activity, and

wherein the second sensor data includes at least one of an orientation and position of the HMD, an ambient pressure, an ambient humidity level, or an ambient light level.

14 . The method of claim 11 , further comprising:

identifying a character content for the virtual fictional character, the character content based on a combination of the user-based context, the ambient-based context, and the application context, the character content comprising an animation content and a speech content.

15 . The method of claim 14 , further comprising:

detecting a change in at least one of the user-based context, the ambient-based context, and the application context; and

adjusting the character content based on the change.

16 . The method of claim 11 , further comprising:

identifying the virtual character based on the application context;

recording an input from the user of the HMD;

communicating the input to a remote server;

receiving audio data in response to the input; and

animating the virtual character based on the audio data.

17 . The method of claim 11 , further comprising:

identifying the virtual fictional character based on a task performed by the user; and

generating a character content for the virtual fictional character, the character content based on a combination of the task, the user-based context, the ambient-based context, and the application context, the character content comprising an animation content and a speech content.

18 . The method of claim 11 , further comprising:

comparing the first sensor data with first reference sensor data for a task performed by the user;

determining the user-based context based on the comparison of the first sensor data with the first reference sensor data;

comparing the second sensor data with second reference sensor data for the task performed by the user; and

determining the ambient-based context based on the comparison of the second sensor data with the second reference sensor data.

19 . The method of claim 18 , wherein the first reference sensor data includes a set of physiological data ranges for the user corresponding to the first set of sensors, a first set of the physiological data ranges corresponding to a first virtual fictional character, and a second set of physiological data ranges corresponding to a second virtual fictional character,

wherein the second reference sensor data includes a set of ambient data ranges for the HMD corresponding to the second set of sensors, a first set of ambient data ranges corresponding to the first virtual fictional character, and a second set of ambient data ranges corresponding to the second virtual fictional character.

20 . A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

measuring first sensor data including an identification of a user of the machine and a biometric state of the user of the machine with a first set of sensors;

measuring second sensor data including a location of the machine and ambient metrics based on the location of the machine with a second set of sensors;

determining a user-based context based on the first sensor data;

determining an ambient-based context based on the second sensor data;

determining, using the one or more processors of the machine, an application context within an AR application implemented by the one or more processors of the machine;

identifying a virtual fictional character based on a combination of the user-based context, the ambient-based context, and the application context; and

displaying the virtual fictional character in a transparent display of the machine.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Oct 26, 2020
From: JEFFERIES FINANCE LLC
To: RPX CORPORATION
Reel/Frame 054486/0422 →
PATENT SECURITY AGREEMENT Recorded Aug 14, 2020
From: RPX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 053498/0095 →
RELEASE OF SECURITY INTEREST Recorded Aug 14, 2020
From: AR HOLDINGS I, LLC
To: DAQRI, LLC
Reel/Frame 053498/0580 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2020
From: DAQRI, LLC
To: RPX CORPORATION
Reel/Frame 053413/0642 →
SECURITY INTEREST Recorded Jun 26, 2019
From: DAQRI, LLC
To: AR HOLDINGS I LLC
Reel/Frame 049596/0965 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2016
From: MULLINS, BRIAN; KAMMERAIT, MATTHEW
To: DAQRI, LLC
Reel/Frame 039295/0381 →