IP Library Granted Patent US 10,325,396
Granted Patent B2
US 10,325,396 · App. 15/432,275 · Granted Jun 18, 2019

Virtual reality presentation of eye movement and eye contact

Inventor: Jeremiah Arthur Grant (Petaluma, CA)
Assignee: LINDEN RESEARCH, INC.
G06T13/40G06F3/011G06F3/013
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,325,396
App. No.
15/432,275
Granted
Jun 18, 2019
Kind
B2
Abstract

A computing system and method to implement a three-dimensional virtual reality world with avatar eye movements without user eye tracking. A position and orientation of a respective avatar in the virtual reality world is tracked to generate a view of the virtual world for the avatar and to present the avatar to others. In response to detecting a predetermined event, the computing system predicts a point (e.g., the eye of another avatar) that is of interest to the respective avatar responsive to the event, and computes, according to an eye movement model, an animation of the eyes of the respective avatar where the gaze of the avatar moves from an initial point to the predicted point and/or its vicinity for a period of time and back to the initial point.

Claims (56)

1. A method implemented in a three-dimensional virtual reality world, the method comprising:

detecting an event related to an avatar having a position and orientation in the virtual reality world, the avatar having at least one eye;

predicting a point of interest to the avatar responsive to the event that triggers eye movement, wherein the point of interest includes an eye of a second avatar and the event corresponds to a real-time communication to or from the second avatar;

computing an animation of the eye of the avatar to move a gaze of the avatar from an initial point to the point of interest without tracking eyes of a user of the avatar, wherein:

the animation moves the gaze of the avatar to the point of interest for a period of time and turns the gaze back to the initial point,

the animation is computed without using user inputs during the animation, and

the animation is computed according to an eye movement model and based on a context determined from the event and personalization parameters of the avatar for which the animation is computed;

switching from a first input mode before the animation to a second input mode during the animation, wherein:

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements;

presenting eye movements of the animation to avatars that have visibility into the animated eye; and

adjusting a view of the virtual world computed for the avatar according to the eye movements of the animation.

2. The method of claim 1 , further comprising:

determining personalization parameters of the eye movement model based on inputs received in the second input mode.

3. The method of claim 2 , wherein the determining personalization parameters is performed using a machine learning technique.

4. The method of claim 3 , wherein the eye movement model is trained via machine learning using eye movements captured in video images of people engaging in social activities.

5. The method of claim 3 , wherein the eye movement model is trained via machine learning using eye movements captured using eye tracking devices of users using the virtual reality world.

6. A method implemented in a three-dimensional virtual reality world, the method comprising:

detecting an event related to an avatar having a position and orientation in the virtual reality world, the avatar having at least one eye;

predicting a point of interest to the avatar responsive to the event that triggers eye movement

computing an animation of the eye of the avatar to move a gaze of the avatar from an initial point to the point of interest without tracking eyes of a user of the avatar;

determining personalization parameters of an eye movement model based on inputs received in a second input mode, wherein the determining personalization parameters is performed using a machine learning technique, and wherein the eye movement model is trained via machine learning using eye movements captured in video images of people engaging in social activities; and

switching from a first input mode before the animation to the second input mode during the animation, wherein:

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements.

7. The method of claim 6 , wherein the animation moves the gaze of the avatar to the point of interest for a period of time and turns the gaze back to the initial point.

8. The method of claim 7 , wherein the animation is computed without using user inputs during the animation.

9. The method of claim 8 , wherein the point of interest includes an eye of a second avatar.

10. The method of claim 9 , wherein the event corresponds to a real-time communication to or from the second avatar.

11. The method of claim 10 , wherein the animation is computed according to an eye movement model.

12. The method of claim 11 , wherein the animation is computed based on a context determined from the event.

13. The method of claim 12 , wherein the animation is computed based on personalization parameters of the avatar for which the animation is computed.

14. The method of claim 13 , further comprising:

presenting eye movements of the animation to avatars that have visibility into the animated eye.

15. A non-transitory computer storage medium storing instructions configured to instruct a computer device to perform a method implemented in a three-dimensional virtual reality world, the method comprising:

detecting an event related to an avatar having a position and orientation in the virtual reality world, the avatar having at least one eye;

predicting a point of interest to the avatar responsive to the event that triggers eye movement;

computing an animation of the eye of the avatar to move a gaze of the avatar from an initial point to the point of interest without tracking eyes of a user of the avatar;

determining personalization parameters of an eye movement model based on inputs received in a second input mode, wherein the determining personalization parameters is performed using a machine learning technique, and wherein the eye movement model is trained via machine learning using eye movements captured using eye tracking devices of users using the virtual reality world; and

switching from a first input mode before the animation to the second input mode during the animation, wherein:

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements.

16. A computing system to implement a three-dimensional virtual reality world, the system comprising:

a server system; and

a data storage device storing:

a three-dimensional model of the virtual reality world; and

avatar models representing residences of the virtual reality world;

wherein the server system generates, from the three-dimensional model of the virtual reality world and the avatar models, data stream to provide views of the virtual reality world to client devices that are connected to the server system via a computer network;

wherein the computing system tracks a position and orientation of a respective avatar in the virtual reality world;

wherein, in response to detecting a predetermined event related to the respective avatar, the computing system predicts a point that is of interest to the respective avatar responsive to the event, and computes an animation of one or more eyes of the respective avatar in which animation a gaze of the respective avatar is moved from an initial point to the predicted point of interest to the avatar; and

wherein the computing system is configured to switch from a first input mode before the animation to a second input mode during the animation, wherein:

when in the first input mode, inputs from one or more devices control the position and orientation of the avatar; and

when in the second input mode, inputs from the one or more devices controls the eye movements;

wherein the animation is computed without input from a client device tracking eyes of a user represented by the respective avatar in the virtual reality world; and

wherein a view of the virtual world generated for the respective avatar is adjusted in accordance with a swift of the gaze of the respective avatar; and during a period of the animation, a user input device that controls the position and orientation of the respective avatar before the period of the animation is automatically reconfigured to control the movement of the gaze during the period of the animation.

17. The computing system of claim 16 , wherein the animation is presented to at least one avatar that has visibility of the one or more eyes of the respective avatar.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2021
From: WOOKEY SEARCH TECHNOLOGIES CORPORATION
To: LINDEN RESEARCH, INC.
Reel/Frame 055660/0179 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2020
From: LINDEN RESEARCH, INC.
To: WOOKEY SEARCH TECHNOLOGIES CORPORATION
Reel/Frame 053327/0454 →
RELEASE OF SECURITY INTEREST Recorded Mar 25, 2020
From: VENTURE LENDING & LEASING VIII, INC.
To: LINDEN RESEARCH, INC.
Reel/Frame 052222/0448 →
SECURITY INTEREST Recorded Mar 12, 2018
From: LINDEN RESEARCH, INC.
To: VENTURE LENDING & LEASING VIII, INC.
Reel/Frame 045569/0228 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2017
From: GRANT, JEREMIAH ARTHUR
To: LINDEN RESEARCH, INC.
Reel/Frame 041254/0837 →
Continuity (1)
Related Publication 20180232928A1 · Aug 16, 2018
Cited By (2)
US 12,210,678 US 12,682,529