IP Library Granted Patent US 8,421,805
Granted Patent B2
US 8,421,805 · App. 11/704,745 · Granted Apr 16, 2013

Smooth morphing between personal video calling avatars

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,421,805
App. No.
11/704,745
Granted
Apr 16, 2013
Kind
B2
Abstract

Dynamic animated avatars selectively morph to reveal or depict the user's identity while simultaneously emulating or tracking active movements associated with the user's verbal stream or perceived movements of the user. A user elects morphing to transition the rendered avatar to another, more revealing avatar, for example from a cartoon caricature to a posed photographic image. Animation processing identifies active movements derived from an input stream to compute animations of the mouth and profile based on speech, text, or captured video from the user. The computed animations appear as active movements to the currently rendered avatar, and emulate the user speaking or moving according to text, audio or video being transmitted. A user maintains an anonymous or posed identity with respect to the receiving party, and chooses to reveal a different avatar depicting a truer identity at the user's discretion, while continuing to display active movements paralleling the user's verbal activities.

Claims (77)

1. A computer implemented method for generating image data associated with a user comprising:

selecting a graphical rendition corresponding to a user that obscures a visual identity of the user;

receiving an indication of active movements applicable to the selected graphical rendition and applicable to the progressed image;

rendering the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements;

selecting a progressed image that is more revealing of the visual identity of the user than the graphical rendition;

receiving an initiator signal provided by the user to begin transforming the rendered image to the progressed image;

in response to receipt of the user-provided initiator signal, generating the progressed image depicting the active movements in each frame of a second series of image frames concurrently with the first series of image frames;

further in response to the user-provided initiator signal, generating a third series of image frames depicting the active movements, each frame in the third series of image frames being generated by combining a frame from the first series of image frames depicting the active movements with a corresponding frame from the second series of image frames depicting the active movements, where the combining is performed by progressive interpolation to cause the third series of image frames to progress from the rendered image to the progressed image while maintaining the active movements; and

rendering the third series of image frames.

2. The method of claim 1 wherein rendering the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements further comprises:

capturing expressive input via an input stream; and

displaying the graphical rendition and animating the rendered image with the first series of image frames using the expressive input, the expressive input corresponding to facial features of spoken verbiage.

3. The method of claim 2 wherein rendering the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements further comprises depicting the active movements by emulating the user performing the speech conveyed by the expressive input, the expressive input contained in an input stream including the indication of active movements.

4. The method of claim 3 further comprising:

performing feature extraction on the input stream to derive facial features;

deriving, from the input stream, facial features including a perspective angle and jaw movements corresponding to the spoken verbiage; and

computing, from the facial features, the active movements.

5. The method of claim 2 wherein capturing expressive input via the input stream includes:

applying feature extraction to an input stream;

deriving an audio stream corresponding to text from the input stream; and

computing mouth movements corresponding to spoken text.

6. The method of claim 1 wherein generating the progressed image further comprises:

selecting a transition image that is more revealing of the visual identity of the user than the graphical rendition;

capturing expressive input via an input stream; and

applying the expressive input to the transition image to form the progressed image in each frame of the second series of image frames.

7. The method of claim 6 wherein combining by progressive interpolation further comprises morphing the rendered image depicting the active movements corresponding to the graphical rendition into another rendered image corresponding to the progressed image.

8. The method of claim 7 wherein morphing the rendered image corresponding to the graphical rendition into another rendered image corresponding to the progressed image includes transitioning the rendered image, the rendered image corresponding to a first base image, to correspond to a second base image.

9. The method of claim 8 wherein the morphing the rendered image corresponding to the graphical rendition into another rendered image corresponding to the progressed image includes responding to DTMF signals in band in an audio component of the input stream.

10. The method of claim 2 wherein capturing expressive input via an input stream includes capturing at least one of verbal signals corresponding to an audio stream or visual input corresponding to captured video signals.

11. The method of claim 10 wherein combining by progressive interpolation further comprises rendering a progression of static images, each of the static images including an incremental change over a previous static image in the progression.

12. The method of claim 11 wherein the graphical rendition is a base image including at least one of:

a caricature; and

a posed photographic image; or a current video feed.

13. The method of claim 1 wherein depicting the active movements further comprises:

extracting features from an input stream, the features indicative of spoken verbiage;

identifying active animations corresponding to the extracted features; and

applying the active animations to the rendered image to generate the appearance of the rendered image verbalizing the spoken language corresponding to the extracted features.

14. The method of claim 13 wherein extracting features from the input stream further comprises:

extracting audio features and video features from the input stream, wherein the extracted features further comprise audio features and video features, further comprising applying active animations to affect mouth, eye and profile appearances of the rendered image.

15. A communications device for generating image data associated with a user comprising:

at least one processor;

an interface to a base image repository being operable to provide to the base image repository a selection of a graphical rendition corresponding to a user that obscures a visual identity of the user;

a selection interface being operable to permit selection of a progressed image that is more revealing of the visual identity of the user than the graphical rendition;

an extractor operable to receive an indication of active movements applicable to the selected graphical rendition and to the selected progressed image;

an animator operable to render the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements, and being further operable to be responsive to an initiator signal provided by the user to generate the progressed image depicting the active movements in each frame of a second series of image frames concurrently with the first series of image frames; and

a morpher operable to be responsive to the user-provided initiator signal to generate a third series of image frames depicting the active movements, each frame in the third series of image frames being generated by combining a frame from the first series of image frames depicting the active movements with a corresponding frame from the second series of image frames depicting the active movements, where the combining is performed by progressive interpolation to cause the third series of image frames to progress from the rendered image to the progressed image while maintaining the active movements.

16. The device of claim 15 wherein the extractor is further operable to:

capture expressive input via an input stream; the animator being responsive to the extractor and operable to display the graphical rendition and animate the rendered image with the first series of image frames using the expressive input, the expressive input corresponding to facial features of spoken verbiage.

17. The device of claim 16 wherein the animator is further operable to:

depict the active movements by emulating user performing the speech conveyed by the expressive input, the expressive input contained in the input stream including the indication of active movements.

18. The device of claim 17 wherein the extractor is further operable to:

perform feature extraction on the input stream to derive facial features;

derive, from the input stream, facial features including a perspective angle and jaw movements corresponding to the spoken verbiage; and

compute, from the facial features, the active movements.

19. The device of claim 16 wherein the extractor further comprises an audio feature extractor operable to:

apply feature extraction to the input stream;

derive an audio stream corresponding to text from the input stream; and

compute mouth movements corresponding to spoken text, further including a video feature extractor operable to extract video features from the input stream; wherein the extracted features further comprise audio features and video features, the video feature extractor being further operable to apply active animations to affect mouth, eye and profile appearances of the rendered image.

20. The device of claim 15 wherein the morpher is further operable to:

receive a selection of a transition avatar that is more revealing of the visual identity of the user than the graphical rendition;

capture expressive input via an input stream; and

apply the expressive input to the transition avatar to form the progressed image in each frame of the second series of image frames.

21. The device of claim 20 wherein the morpher is further operable respond to DTMF signals in band in an audio component of the input stream.

22. The device of claim 16 wherein the extractor further includes an audio feature extractor and a video feature extractor, the audio feature extractor being operable to capture verbal signals corresponding to an audio stream, the video feature extractor operable to capture visual input corresponding to captured video signals.

23. A computer program product having a non-transitory computer readable medium operable to store a set of encoded instructions for execution by a processor responsive to the instructions for generating image data associated with a user, the set of encoded instructions comprising:

computer program code for selecting a graphical rendition corresponding to a user that obscures a visual identity of the user;

computer program code for capturing expressive input via an input stream;

computer program code for rendering the graphical rendition as a rendered image using the expressive input in each frame of a first series of image frames, the expressive input corresponding to facial features of spoken verbiage;

computer program code for selecting a progressed image that is more revealing of the visual identity of the user than the graphical rendition;

computer program code for receiving an indication provided by the user to transform the rendered image to the progressed image;

computer program code for responding to receiving the indication by generating the progressed image using the expressive input in each frame of a second series of image frames concurrently with the first series of image frames;

computer program code for responding to receiving the indication by generating a third series of image frames that maintain the usage of the expressive input, each frame in the third series of image frames being generated by combining a frame from the first series of image frames with a corresponding frame from the second series of image frames, where the combining is performed by progressive interpolation to generate a progression of resulting morphed image frames that maintain the usage of the expressive input while gradually transforming the rendered image to the progressed image; and

computer program code for rendering the progression of resulting morphed image frames.

24. The device of claim 15 wherein the morpher is further operable to:

receive extracted features from an input stream, the extracted features being indicative of spoken verbiage;

identify active animations corresponding to the extracted features; and

apply the active animations to the rendered image to generate an appearance of the rendered image verbalizing the spoken verbiage corresponding to the extracted features.

Assignments (7)
SECURITY AGREEMENT Recorded Jun 30, 2015
From: DIALOGIC (US) INC.; DIALOGIC INC.; DIALOGIC US HOLDINGS INC.; DIALOGIC DISTRIBUTION LIMITED; DIALOGIC MANUFACTURING LIMITED; DIALOGIC CORPORATION; DIALOGIC GROUP INC.
To: SILICON VALLEY BANK
Reel/Frame 036037/0165 →
RELEASE OF SECURITY INTEREST Recorded Nov 25, 2014
From: OBSIDIAN, LLC
To: DIALOGIC CORPORATION, F/K/A EICON NETWORKS CORPORATION; DIALOGIC INC.; DIALOGIC (US) INC., F/K/A DIALOGIC INC. AND F/K/A EICON NETWORKS INC.; DIALOGIC DISTRIBUTION LIMITED, F/K/A EICON NETWORKS DISTRIBUTION LIMITED; DIALOGIC MANUFACTURING LIMITED, F/K/A EICON NETWORKS MANUFACTURING LIMITED; DIALOGIC US HOLDINGS INC.; DIALOGIC RESEARCH INC., F/K/A EICON NETWORKS RESEARCH INC.; DIALOGIC JAPAN, INC., F/K/A CANTATA JAPAN, INC.; CANTATA TECHNOLOGY, INC.; EAS GROUP, INC.; SHIVA (US) NETWORK CORPORATION; EXCEL SWITCHING CORPORATION; EXCEL SECURITIES CORPORATION; CANTATA TECHNOLOGY INTERNATIONAL, INC.; BROOKTROUT NETWORKS GROUP, INC.; BROOKTROUT TECHNOLOGY, INC.; SNOWSHORE NETWORKS, INC.; BROOKTROUT SECURITIES CORPORATION
Reel/Frame 034468/0654 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Dec 23, 2008
From: DIALOGIC CORPORATION
To: OBSIDIAN, LLC
Reel/Frame 022024/0274 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2008
From: NMS COMMUNICATIONS CORPORATION
To: DIALOGIC CORPORATION
Reel/Frame 021985/0001 →
ACKNOWLEDGMENT OF RELEASE OF INTELLECTUAL PROPERTY COLLATERAL Recorded Dec 15, 2008
From: SILICON VALLEY BANK
To: LIVEWIRE MOBILE, FORMERLY KNOWN AS NMS COMMUNICATIONS CORPORATION
Reel/Frame 021976/0628 →
SECURITY AGREEMENT Recorded Nov 19, 2008
From: NMS COMMUNICATIONS CORPORATION
To: SILICON VALLEY BANK
Reel/Frame 021861/0051 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2008
From: BISHOP, WENDELL E.
To: NMS COMMUNICATIONS CORPORATION
Reel/Frame 021487/0537 →