IP Library Granted Patent US 9,424,678
Granted Patent B1
US 9,424,678 · App. 13/968,489 · Granted Aug 23, 2016

Method for teleconferencing using 3-D avatar

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,424,678
App. No.
13/968,489
Granted
Aug 23, 2016
Kind
B1
Abstract

A method for implementing teleconferences when at least one participant receives 3-D data. A data rendering device presents data in a 3-D format or in pseudo 3-D format. A 3-D image is formed on a user computer system. The 3-D presentation is calculated by a local computer system. A block of user avatars is formed on the local computer system for all teleconference participants (including a local participant). The participant avatar includes a kinetic model (joints, muscles, body parts, etc.). The avatar includes a participant's behavior model (gestures, fingers, mimics, etc). The avatar also includes an avatar skin. The avatar skin includes a stable (unchangeable) part of the participant containing face and visible body parts, as well as modifiable parts (e.g., clothes, shoes, accessories, etc.).

Claims (42)

1. A system for providing a virtual conference environment using 3-D avatars, the system comprising:

a plurality of remotely controlled video cameras configured to record image data of conference participants, respectively;

a plurality of directional microphones configured to record audio data of the conference participants, respectively;

a plurality of speakers for reproducing the recorded audio data of the conference participants;

at least one kinetic sensor configured to detect and record movements of at least one of the conference participants, respectively, in a form of a kinetic data;

a virtual reality projector configured to render 3-D virtual data to the participants, the 3-D virtual data providing the virtual conference environment that includes each of the conference participants illustrated as the 3-D avatars; and

a computer processor configured to generate the 3-D virtual data by combining the image data with recorded audio data and the kinetic data;

wherein the 3-D avatars are stored in at least one avatar database and accessed by the computer processor when creating the 3-D virtual data,

wherein the computer processor is further configured to: (i) extract differential and behavior parameters from the recorded image data for a selected 3-D avatar, (ii) optimize the recorded image data for the selected 3-D avatar for a transmission channel based on capacity and latency of the transmission channel, (iii) transfer the optimized data to a client associated with the selected 3-D avatar based on the capacity and latency, and (iv) apply the differential and behavior parameters to the selected 3-D avatar illustrated in the virtual conference environment to the client associated with the selected 3-D avatar,

wherein, when at least one additional conference participant attempts to join the virtual conference environment, the computer processor accesses the at least one avatar database to identify a 3-D avatar associated with the at least one additional conference participant, and

wherein, if the at least one avatar database does not have the 3-D avatar associated with the at least one additional conference participant, the computer processor is configured to access a photograph or video of the at least one additional conference participant from a source other than the at least one avatar database and generate the 3-D avatar associated with the at least one additional conference participant that is displayed in the virtual conference environment.

2. The system of claim 1 , wherein the avatar database is located in a cloud.

3. The system of claim 1 further comprising at least one Doppler sensor configured to acquire the kinetic data.

4. The system of claim 1 , wherein the kinetic data is acquired using traceable markers attached to body parts of the conference participants.

5. The system of claim 1 , wherein the 3-D avatars are provided to remote server over a cloud.

6. The system of claim 1 , wherein the 3-D avatars are displayed using at least one of:

a hologram;

virtual reality glasses;

a 3-D monitor; and

steam cloud images.

7. A method for providing a virtual conference environment using 3-D avatars, the method comprising:

capturing, by a plurality of remotely controlled video cameras, a video data of conference participants, respectively;

capturing, by a plurality of directional microphones, audio data of the conference participants, respectively;

detecting and recording, by at least one kinetic sensor, movements of at least one of the conference participants, respectively, as kinetic data;

generating 3-D virtual data by combining the captured video data with the captured audio data and with the recorded kinetic data;

rendering, by a virtual reality projector, the 3-D virtual data providing the virtual conference environment including each of the conference participants illustrated as the 3-D avatars;

storing the 3-D avatars in at least one avatar database;

extracting differential and behavior parameters from the captured data for a selected 3-D avatar;

optimizing the captured data for the selected avatar for a transmission channel based on capacity and latency of the transmission channel;

transferring the optimized data to a client based on the capacity and latency;

applying the differential and behavior parameters to the avatar illustrated in the virtual conference environment to the client;

when at least one additional conference participant attempts to join the virtual conference environment, accessing the at least one avatar database to identify a 3-D avatar associated with the at least one additional conference participant;

if the at least one avatar database does not have the 3-D avatar associated with the at least one additional conference participant, accessing a photograph or video of the at least one additional conference participant from a source other than the at least one avatar database; and

generating the 3-D avatar associated with the at least one additional conference participant that is displayed in the virtual conference environment.

8. The method of claim 7 , further comprising adding a skin to the 3-D avatars.

9. The method of claim 7 , further comprising masking lags in data transmission through the channel by applying behavior patterns.

10. The method of claim 9 , wherein the behavior patterns are predefined and calculated during teleconference to avoid artifacts.

11. The method of claim 9 , wherein the behavior patterns are extracted from prior data transmission translation and applied to any combination of micro-movement of parts of the 3-D avatar.

12. The method of claim 11 , wherein the behavior patterns are applied to unintended gestures and movements to keep the 3-D avatar's appearance live.

13. The method of claim 7 , further comprising comparing current images of the participants with the 3-D avatars and correcting the avatars if the differences are detected.

14. The method of claim 7 further comprising generating artificial intelligence models for the 3-D avatars.

15. The method of claim 7 , further comprising authenticating the 3-D avatars and changing an appearance of the 3-D avatars based on a successful authentication.

Assignments (6)
REAFFIRMATION AGREEMENT Recorded Aug 28, 2022
From: ACRONIS AG; ACRONIS INTERNATIONAL GMBH; ACRONIS SCS, INC.; ACRONIS, INC.; GROUPLOGIC, INC.; NSCALED INC.; ACRONIS MANAGEMENT LLC; 5NINE SOFTWARE, INC.; ACRONIS GERMANY GMBH; ACRONIS NETHERLANDS B.V.; ACRONIS BULGARIA EOOD; DEVICELOCK, INC.; DEVLOCKCORP LTD; ACRONIS INC.
To: MIDCAP FINANCIAL TRUST
Reel/Frame 061330/0818 →
SECURITY INTEREST Recorded Dec 19, 2019
From: ACRONIS INTERNATIONAL GMBH
To: MIDCAP FINANCIAL TRUST
Reel/Frame 051418/0119 →
RELEASE OF SECURITY INTEREST Recorded Oct 21, 2019
From: OBSIDIAN AGENCY SERVICES, INC.
To: ACRONIS INTERNATIONAL GMBH; GROUPLOGIC, INC.
Reel/Frame 050783/0893 →
SECURITY INTEREST Recorded Jul 26, 2017
From: ACRONIS INTERNATIONAL GMBH; GROUPLOGIC, INC.
To: OBSIDIAN AGENCY SERVICES, INC., AS COLLATERAL AGENT
Reel/Frame 043350/0186 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2016
From: RUNA CAPITAL
To: ACRONIS INTERNATIONAL GMBH
Reel/Frame 037813/0230 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 16, 2013
From: ENAKIEV, ANTON; BELOUSSOV, SERGUEI M.; TSIBROVSKI, JURI V.; TORMASOV, ALEXANDER G.; PROTASSOV, STANISLAV S.
To: RUNA CAPITAL
Reel/Frame 031023/0567 →