IP Library Granted Patent US 11,943,563
Granted Patent B2
US 11,943,563 · App. 17/424,680 · Granted Mar 26, 2024

Videoconferencing terminal and method of operating the same

Inventors: Ola Wassvik (Lammhult, SE); Mats Peter Wallander (Lund, SE)
Assignee: FlatFrog Laboratories, AB
H04N7/144G06T5/005H04N7/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,943,563
App. No.
17/424,680
Granted
Mar 26, 2024
Kind
B2
Abstract

A method of videoconferencing comprises displaying an image of a remote user on a display and capturing an image of a local user at a user position in front of the display. The at least one camera is located at a camera position behind the display. The method comprises modifying an image to be displayed based on the camera position of the at least one camera with respect to the display and based on the user position of the local user with respect to the display.

Claims (38)

1. A method of videoconferencing comprising:

displaying an image of a remote user on a display, wherein the display comprises an LED matrix;

capturing an image of a local user at a user position in front of the display, with at least one camera being located at a camera position behind the display;

calibrating the image of the local user by capturing a camera image including a local user captured image of the local user together with pixel artifacts and occlusion artifacts by:

determining one or more pixel artifacts captured by the at least one camera by receiving light from pixels of the display,

compensating the captured camera image to remove the determined one or more pixel artifacts,

determining one or more occlusion artifacts from one or more display elements, and

compensating the captured camera image to remove the one or more occlusion artifacts,

wherein the compensated captured camera image comprises the local user captured image; and

modifying the image of the remote user to be displayed based on the camera position of the at least one camera with respect to the display and based on the user position of the local user with respect to the display.

2. A method according to claim 1 wherein the method comprises determining a position of an eye of the local user with respect to the display.

3. A method according to claim 2 wherein the method comprises determining an axis of the at least one camera based on the position of the eyes of the local user.

4. A method according to claim 1 wherein the method comprises determining a position of an eye of the remote user with respect to the display.

5. A method according to claim 4 wherein the method comprises determining an offset between the axis of the camera and an eye of the remote user in a displayed image.

6. A method according to claim 5 wherein the modifying comprises translating the image of the remote user to be displayed such that the displayed eyes of the remote user intersect with the axis.

7. A method according to claim 1 wherein occluding display elements are out of focus.

8. A method according to claim 1 wherein the user position of the user and/or the camera position of the at least one camera is moveable with respect to the display.

9. A method according to claim 1 wherein the at least one camera is one or more of the following: an RGB camera or an infrared camera.

10. A method according to claim 1 wherein the display is transmissive to electromagnetic radiation.

11. A method according to claim 1 wherein one or more of the steps is carried out during calibration and/or one or more of the steps is carried out during operation.

12. The method according to claim 1 , wherein the display is an OLED display.

13. The method according to claim 1 , further comprising determining a maximum pixel level of a particular pixel in the LED matrix, and, for each other pixel in the LED matrix, dividing its pixel value by the maximum pixel value to get an occlusion factor for each particular pixel.

14. A videoconferencing terminal comprising:

a display for displaying an image of a remote user, wherein the display comprises an LED matrix;

at least one camera for capturing an image of a local user at a user position in front of the display, the camera being located at a camera position behind the display; and

a controller configured to modify the image of the remote user to be displayed based on the camera position of the at least one camera with respect to the display and based on the user position of the local user with respect to the display,

wherein the controller is further configured to calibrate the image of the local user by capturing a camera image including a local user captured image of the local user together with pixel artifacts and/or occlusion artifacts, wherein the calibration comprises:

determining one or more pixel artifacts captured by the at least one camera by receiving light from pixels of the display,

compensating the captured camera image to remove the determined one or more pixel artifacts,

determining one or more occlusion artifacts from one or more display elements, and

compensating the captured camera image to remove the one or more occlusion artifacts, wherein the compensated captured camera image comprises the local user captured image.

15. A method of videoconferencing comprising:

displaying an image of a remote user on a display, wherein the display comprises LED matrix;

capturing an image of a local user at a user position in front of the display, with at least one camera being located at a camera position behind the display;

calibrating the image of the local user by capturing a camera image including a local user captured image of the local user together with pixel artifacts and/or occlusion artifacts by:

determining one or more pixel artifacts captured by the at least one camera by receiving light from pixels of the display, and

compensating the captured camera image to remove the determined one or more pixel artifacts; and

modifying the image of the remote user to be displayed based on the camera position of the at least one camera with respect to the display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 16, 2021
From: WASSVIK, OLA; WALLANDER, MATS PETER
To: FLATFROG LABORATORIES AB
Reel/Frame 057189/0417 →
Priority Claims (1)
SE 1930022-7 · Jan 25, 2019 · national
Continuity (1)
Related Publication 20220109809A1 · Apr 7, 2022
Cited By (5)
US 12,282,653 US 12,461,630 US 12,524,116 US 12,524,117 US 12,704,928