IP Library Granted Patent US 11,196,972
Granted Patent B2
US 11,196,972 · App. 15/764,696 · Granted Dec 7, 2021

Presence camera

Inventor: Cevat Yerli (Frankfurt am Main, DE)
Assignee: TMRW Foundation IP S. À R.L.
H04N13/111H04N13/194H04N13/243H04N13/344H04N21/00H04N21/21805H04N21/6587
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,196,972
App. No.
15/764,696
Granted
Dec 7, 2021
Kind
B2
Abstract

A device is disclosed which comprises a media interface configured to provide a plurality of media streams, each media stream being captured using a camera of at least one array of cameras, an input interface configured to receive a device position associated with an output device, and a processor configured to generate from the plurality of media streams an output stream adjusted to the device position.

Claims (36)

1. A device comprising:

a media interface configured to provide a plurality of media streams, each media stream being captured with a full 6 degrees of freedom parallax using a camera of at least one array of cameras;

an input interface configured to receive a viewing position and orientation of a display device, the viewing position and orientation having 6 degrees of freedom; and

a processor configured to generate from the plurality of media streams a plurality of output streams based on the viewing position and orientation of the display device, wherein each output stream reflects the viewing position and orientation adjusted by an offset, wherein according to a current viewing position and orientation of the display device, the processor is further configured to warp between the plurality of output streams to an output stream nearest to the current viewing position and orientation of the display device.

2. The device according to claim 1 wherein the at least one array of cameras is configured to capture a scene, each camera of the at least one array being configured to capture at least a part of the scene according to camera parameters.

3. The device according to claim 1 wherein two or more media streams are determined as the n nearest media streams according to a field of view of the display device and parts of the scene captured by the cameras in the media streams.

4. The device according to claim 1 wherein at least a part of the plurality of media streams are pre-recorded media streams or live media streams.

5. The device according to claim 1 wherein the processor is further configured to combine at least one of the plurality of output streams with at least one computer graphics object, the device further comprising an engine communicatively coupled to the processor, said engine configured to provide the at least one computer graphics object according to the viewing position and orientation of the display device.

6. The device according to claim 1 , wherein the processor is further configured to decode the plurality of media streams, wherein at least some of the plurality of media streams are compressed based on another one of the plurality of media streams.

7. The device according to claim 1 wherein the at least one array of cameras includes at least one lens with an ultra-wide angle.

8. The device according to claim 1 wherein at least one camera of the at least one array of cameras is a stereoscopic camera or a camera configured to capture depth information.

9. The device according to claim 1 wherein the cameras of the at least one array of cameras are arranged on a surface according to a pattern, wherein the surface is a flat or curved surface.

10. The device according to claim 1 comprising two or more arrays of cameras, wherein the arrays are rotated with respect to each other by a predetermined angle.

11. An output device comprising:

at least one display; and

means to detect a viewing position of the output device, the output device further comprising and being connected to a device according to claim 1 .

12. The device according to claim 1 wherein at least one of the plurality of output streams includes a motion parallax effect adapted to motion of the display device.

13. The device according to claim 1 wherein the display device comprises a wearable display device.

14. A method comprising:

providing a plurality of media streams, each media stream being captured with a full 6 degrees of freedom parallax using a camera of at least one array of cameras;

receiving a viewing position and orientation of a display device, the viewing position and orientation having 6 degrees of freedom;

generating from the plurality of media streams a plurality of output streams based on the viewing position and orientation of the display device, wherein each output stream reflects the viewing position and orientation of the display device adjusted by an offset; and

according to a current viewing position and orientation of the display device, warping between the plurality of output streams to an output stream nearest to the current viewing position and orientation of the display device.

15. The method according to claim 14 further comprising determining two or more media streams as n nearest media streams according to a field of view of the display device and parts of the scene captured by the cameras in the media streams.

16. The method according to claim 14 further comprising:

combining at least one of the plurality of output streams with at least one computer graphics object; and

rendering the at least one computer graphics object according to the viewing position and orientation of the display device.

17. The method according to claim 14 further comprising decoding the plurality of media streams, wherein at least some of the plurality of media streams are compressed based on another one of the plurality of media streams.

18. The method according to claim 14 wherein at least one of the plurality of output streams includes a motion parallax effect adapted to motion of the display device.

19. The method according to claim 14 wherein the display device comprises a wearable display device.

20. A computing device comprising one or more processors and memory, the memory having stored thereon executable instructions configured to cause the one or more processors to perform steps including:

obtaining a plurality of media streams, each media stream being captured with a full 6 degrees of freedom parallax using a camera of at least one array of cameras;

receiving a position and orientation of a virtual reality headset in motion, the position and orientation having 6 degrees of freedom;

generating from the plurality of media streams a plurality of output streams based on the viewing position and orientation of the virtual reality headset, wherein each output stream reflects the viewing position and orientation of the virtual reality headset adjusted by an offset; and

according to a current viewing position and orientation of the virtual reality headset, warping between the plurality of output streams to an output stream nearest to the current viewing position and orientation of the virtual reality headset.

21. The computing device according to claim 20 wherein at least one of the plurality of output streams includes a motion parallax effect adapted to motion of the virtual reality headset.

Assignments (4)
CHANGE OF NAME Recorded Apr 18, 2025
From: TMRW FOUNDATION IP S.À R.L.
To: TMRW GROUP IP
Reel/Frame 070891/0012 →
CHANGE OF NAME Recorded May 24, 2019
From: CINEVR EUROPE S. À R.L.
To: TMRW FOUNDATION IP S. À R.L.
Reel/Frame 049287/0827 →
CHANGE OF NAME Recorded May 10, 2018
From: CALAY VENTURE S. À R.L.
To: CINEVR EUROPE S. À R.L.
Reel/Frame 046677/0276 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2018
From: YERLI, CEVAT
To: CALAY VENTURE S.À R.L.
Reel/Frame 045522/0038 →
Priority Claims (2)
EP 15187627 · Sep 30, 2015 · regional
EP 16154501 · Feb 5, 2016 · regional
Continuity (1)
Related Publication 20180288393A1 · Oct 4, 2018