IP Library › Granted Patent US 9,386,268
Granted Patent B2
US 9,386,268 · App. 13/996,009 · Granted Jul 5, 2016

Communication using interactive avatars

Inventors: Xiaofeng Tong (Beijing, CN); Wenlong Li (Beijing, CN); Yangzhou Du (Beijing, CN); Wei Hu (Beijing, CN); Yimin Zhang (Beijing, CN)
Assignee: Intel Corporation
H04N7/147G06T13/40H04M1/72555
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,386,268
App. No.
13/996,009
Granted
Jul 5, 2016
Kind
B2
Abstract

Generally this disclosure describes a video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar; initiating communication; detecting a user input; identifying the user input; identifying an animation command based on the user input; generating avatar parameters; and transmitting at least one of the animation command and the avatar parameters.

Claims (95)

1. A system, comprising:

a user input device configured to capture a user input;

communication circuitry configured to transmit and receive information;

a microphone configured to capture sound and convert the captured sound into a corresponding audio signal; and

one or more non-transitory storage memories having stored thereon, individually or in combination, instructions that when executed by one or more processors result in the following operations comprising:

selecting an avatar;

receiving at least one image of a user;

passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;

detecting a user input with said user input device, said user input comprising at least one of a touch and a gesture;

determining one or more animation commands associated with a user input identifier corresponding to a detected user input;

determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;

modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar;

transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device;

capturing user speech and converting the user speech into a corresponding user speech signal;

transforming the user speech signal into an avatar speech signal; and

transmitting the avatar speech signal to the remote device.

2. The system of claim 1 , further comprising a camera configured to capture images, wherein the instructions that when executed by one or more processors result in the following additional operations:

capturing the at least one image of the user;

performing facial detection on said at least one image to detect a face in the image;

extracting features from the face; and

passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.

3. The system of claim 1 , further comprising a display, wherein the instructions that when executed by one or more processors result in the following additional operations:

displaying said avatar;

receiving at least one of a remote animation command and remote avatar parameters; and

passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.

4. The system of claim 1 , further comprising a speaker configured to convert an audio signal into sound, wherein the instructions that when executed by one or more processors result in the following additional operations:

receiving a remote avatar speech signal; and

converting the remote avatar speech signal into avatar speech.

5. The system of claim 1 , wherein:

the user input device is a depth camera; and

the user input is a gesture detected by said depth camera.

6. The system of claim 1 , wherein:

the user input device is a touch-sensitive display;

the user input is a touch event; and

said touch event comprises at least one of a touch type and a touch location.

7. The system of claim 1 , wherein the transforming comprises at least one of pitch shifting and time stretching.

8. A method, comprising:

selecting an avatar;

receiving at least one image of a user;

passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;

detecting a user input with a user input device, said user input comprising at least one of a touch and a gesture;

determining one or more animation commands associated with a user input identifier corresponding to a detected user input;

determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;

modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar;

transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device;

capturing user speech and converting the user speech into a corresponding user speech signal;

transforming the user speech signal into an avatar speech signal; and

transmitting the avatar speech signal to the remote device.

9. The method of claim 8 , further comprising:

capturing the at least one image of the user;

performing facial detection on said at least one image to detect a face in the image;

extracting features from the face; and

passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.

10. The method of claim 8 , further comprising:

displaying said avatar;

receiving at least one of a remote animation command and remote avatar parameters; and

passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.

11. The method of claim 8 , further comprising:

receiving a remote avatar speech signal; and

converting the remote avatar speech signal into avatar speech.

12. The method of claim 8 , wherein:

the user input device is a depth camera; and

the user input is a gesture detected by said depth camera.

13. The method of claim 8 , wherein:

The user input device is a touch-sensitive display;

the user input is a touch event; and

the touch event comprises at least one of a touch type and a touch location.

14. The method of claim 8 , wherein the transforming comprises at least one of pitch shifting and time stretching.

15. A system comprising one or more non-transitory storage memories having stored thereon, individually or in combination, instructions that when executed by one or more processors result in the following operations comprising:

selecting an avatar;

receiving at least one image of a user;

passively animating the avatar based at least in part on facial mapping of the at least one image, so as to produce a passively animated avatar for display on a remote device, wherein the passively animated avatar mimics motion of a body part of a user;

detecting a user input, said user input comprising at least one of a touch and a gesture;

determining one or more animation commands associated with a user input identifier corresponding to a detected user input;

determining an interactive animation for said passively animated avatar based at least in part on said one or more animation commands;

modifying said passively animated avatar with said interactive animation, so as to produce an interactively animated avatar by deforming at least a portion of said passively animated avatar;

transmitting a signal to said remote device, said signal configured to cause said interactively animated avatar to be displayed on said remote device;

capturing user speech and converting the user speech into a corresponding user speech signal;

transforming the user speech signal into an avatar speech signal; and

transmitting the avatar speech signal to the remote device.

16. The system of claim 15 , wherein the instructions that when executed by one or more processors result in the following additional operations:

capturing the at least one image of the user;

performing facial detection on said at least one image to detect a face in the image;

extracting features from the face; and

passively animating the avatar based at least in part on extracted features from said face, such that said passively animated avatar mimics motion of at least a portion of said face.

17. The system of claim 15 , wherein the instructions that when executed by one or more processors result in the following additional operations:

displaying said avatar;

receiving at least one of a remote animation command and remote avatar parameters; and

passively animating said avatar at least in part based on at least one of the remote animation command and the remote avatar parameters.

18. The system of claim 15 , wherein the instructions that when executed by one or more processors result in the following additional operations:

receiving a remote avatar speech signal; and

converting the remote avatar speech signal into avatar speech.

19. The system of claim 15 , wherein the user input is a gesture detected by a depth camera.

20. The system of claim 15 , wherein the user input is a touch event detected by a touch-sensitive display, the touch even comprising at least one of a touch type and a touch location.

21. The system of claim 15 , wherein the transforming comprises at least one of pitch shifting and time stretching.

Continuity (1)
Related Publication 20140152758A1 · Jun 5, 2014