IP Library Granted Patent US 11,792,367
Granted Patent B2
US 11,792,367 · App. 17/249,468 · Granted Oct 17, 2023

Method and system for virtual 3D communications

Inventors: Ran Oz (Maccabim, IL); Amir Bassan-Eskenazi (Los Altos, CA); Yuval Gronau (Ramat Hasharon, IL); Michael Rabinovich (Tel Aviv, IL); Osnat Goren-Peyser (Tel Aviv, IL); Tal Perl (Los Altos, CA); Erez Posner (Rehovot, IL)
Assignee: TRUE MEETING INC.
H04N7/157G06F3/013G06N3/04G06N3/045G06T7/11G06T7/70G06T15/04G06T15/20G06T15/205G06T17/20G06T19/20H04N7/144H04N7/147H04N7/152G06T19/00G06T2200/08G06T2207/30201G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,792,367
App. No.
17/249,468
Granted
Oct 17, 2023
Kind
B2
Abstract

A method for conducting a three dimensional (3D) video conference between multiple participants, the method may include receiving direction of gaze information regarding a direction of gaze of each participant within a representation of a virtual 3D video conference environment that is associated with the participant; determining, for each participant, updated 3D participant representation information within the virtual 3D video conference environment, that reflects the direction of gaze of the participant; and generating, for at least one participant, an updated representation of virtual 3D video conference environment, the updated representation of virtual 3D video conference environment represents the updated 3D participant representation information for at least some of the multiple participants.

Claims (32)

1. A method for conducting a three dimensional (3D) video conference between multiple participants, the method comprises:

receiving direction of gaze information regarding a direction of gaze of a first participant of the multiple participants;

receiving one or more image acquisition parameters related to the first participant;

determining, based on the direction of gaze of the first participant, that the first participant is looking at another avatar that represents another participant of the multiple participants, the other avatar is displayed to the first participant within a representation of a virtual 3D video conference environment (V3VCE) that is associated with the first participant; wherein for each participant, a direction of gaze of the participant is determined based on images acquired by a single camera of the participant; wherein images of the multiple participants are acquired by multiple cameras; wherein the multiple participants are associated with multiple representations of the V3VCE;

determining updated 3D participant representation information within the V3VCE, wherein the updated 3D participant representation information reflects the direction of gaze of the participant and the one or more image acquisition parameters; and

updating the multiple representations of the V3VCE with the updated 3D participant representation information so that each one of the multiple representations of the V3VCE comprises avatars of the multiple participants, the avatars of the multiple participants comprise a first avatar that represents the first participant and another avatar that represents the other participant, wherein the first avatar gazes at the other avatar; wherein locations the avatars of the multiple participants and a direction of gaze of the first avatar are the same in all the representations of the V3VCE.

2. The method according to claim 1 , wherein the updating is followed by displaying, by a device of a participant of the multiple participants, an updated representation of the V3VCE that is associated with the participant.

3. The method according to claim 1 , wherein the multiple participants are associated with multiple participant devices, wherein the receiving and determining are executed by only a part of the multiple participant devices.

4. The method according to claim 1 , wherein the multiple participants are associated with multiple participant devices, wherein the receiving and determining are executed by a only a part of the multiple participant device and by a computerized system that differs from any of the multiple participant devices.

5. The method according to claim 1 , comprising determining a field of view of a third participant within the V3VCE, and setting a third updated representation of the virtual 3D video conference environment that is sent to a third participant device to reflect the field of view of the third participant.

6. The method according to claim 1 , wherein the 3D participant representation information comprises a 3D model and one or more texture maps.

7. The method according to claim 1 , wherein for each participant, the updated representation of the V3VCE associated with the participant comprises an avatar per participant of the at least some of the multiple participants.

8. The method according to claim 7 , wherein the direction of gaze of the first avatar within the V3VCE represents a spatial relationship between the direction of gaze of the first participant and a representation of the V3VCE displayed to the first participant.

9. The method according to claim 1 , wherein the first avatar is generated in part based on received information retrieved from a social network.

10. The method according to claim 1 , wherein the method comprises generating a 3D model and one or more texture maps of 3D participant representation information of a participant.

11. The method according to claim 10 , wherein the updating is based on images of the participant that were acquired under different circumstances.

12. The method according to claim 11 , wherein the different circumstances comprise different viewing directions of a camera that acquired the images, different poses, and different expressions of the participant.

13. The method according to claim 1 , comprising estimating 3D participant representation information of one or more hidden areas of a face of a participant that are located outside a field of view of a camera that captures at least one visible area of the face of the participant.

14. The method according to claim 13 , comprising estimating 3D model hidden areas and one or more hidden parts texture maps.

15. The method according to claim 1 , wherein the determining, for each participant, of the updated 3D participant representation information comprises changing an illumination condition.

16. The method according to claim 1 , comprising acquiring, by a camera of a participant, an image of a face of the participant, finding teeth of the participant, and determine an exposure of the camera based on a brightness of the teeth.

17. The method according to claim 1 , wherein for each participant, the updated representation of the V3VCE illustrates all participants.

18. The method according to claim 1 , wherein the direction of gaze information of each participant is obtained by a participant device that is selected from a desktop, a laptop, a mobile phone and a tablet.

19. The method according to claim 1 , wherein the direction of gaze information of each participant is obtained based on a front facing image of a head of each participant.

20. The method according to claim 1 wherein for each participant other than the first participant, a direction of gaze of an avatar that represents the participant does not reflect an optical axis of a camera that acquired images of the participant.

21. The method according to claim 1 wherein for each participant the updated representation of the V3VCE that is associated with the participant is rendered from a point of view of a virtual camera associated with the participant.

22. At least one non-transitory computer readable medium for conducting a three dimensional (3D) video conference between multiple participants, the at least non-transitory computer readable medium that stores instructions for:

receiving direction of gaze information regarding a direction of gaze of a first participant of the multiple participants;

receiving one or more image acquisition parameters related to the first participant;

determining, based on the direction of gaze of the first participant, that the first participant is looking at another avatar that represents another participant of the multiple participants, the other avatar is displayed to the first participant within a representation of a virtual 3D video conference environment (V3VCE) that is associated with the first participant; wherein for each participant, a direction of gaze of the participant is determined based on images acquired by a single camera of the participant; wherein images of the multiple participants are acquired by multiple cameras; wherein the multiple participants are associated with multiple representations of the V3VCE;

determining updated 3D participant representation information within the V3VCE, wherein the updated 3D participant representation information reflects the direction of gaze of the participant and the one or more image acquisition parameters; and

updating the multiple representations of the V3VCE with the updated 3D participant representation information so that each one of the multiple representations of the V3VCE comprises avatars of the multiple participants, the avatars of the multiple participants comprise a first avatar that represents the first participant and another avatar that represents the other participant, wherein the first avatar gazes at the other avatar; wherein locations the avatars of the multiple participants and a direction of gaze of the first avatar are the same in all the representations of the V3VCE.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2025
From: TRUEMEETING, INC.
To: CAVENDISH CAPITAL LLC
Reel/Frame 070723/0760 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2022
From: POSNER, EREZ
To: TRUE MEETING INC.
Reel/Frame 061534/0424 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2021
From: OZ, RAN; BASSAN-ESKENAZI, AMIR; GRONAU, YUVAL; RABINOVICH, MICHAEL; GOREN-PEYSER, OSNAT
To: TRUE MEETING INC.
Reel/Frame 056192/0164 →
Continuity (4)
Provisional Application 63199014 · Dec 1, 2020
Provisional Application 63081860 · Sep 22, 2020
Provisional Application 63023836 · May 12, 2020
Related Publication 20210360196A1 · Nov 18, 2021
Cited By (2)
US 12,542,869 US 12,659,426