Granted Patent B2

US 12,395,618 · App. 18/205,527 · Granted Aug 19, 2025

Inventor: Nicolas Dahlquist (Redwood City, CA)

Assignee: LEIA SPV LLC

H04N13/351G06T1/20G06T5/50G06T2207/20221

Abstract

Claims (50)

1. A computer-implemented method of real-time multiview video conversion, the method comprising:

receiving a video stream comprising two-dimensional (2D) frames by a computing device, each 2D frame corresponding to a respective 2D video timestamp;

identifying a camera baseline and a center viewpoint by the computing device, wherein the camera baseline defines an interval between views of a multiview frame, and the center viewpoint is a midpoint in a range of views of the multiview frame;

determining a respective target timestamp for each view of the multiview frame based on the camera baseline and the center viewpoint by the computing device;

using the determined target timestamps, generating views of the multiview frame from respective subsets of 2D frames having 2D video timestamps adjacent to the target timestamp for each respective view by the computing device; and

rendering a multiview video for display by the computing device, the multiview video comprising the multiview frame.

2. The method of claim 1 , further comprising decoding the video stream to extract the 2D frames in a graphics processing unit memory of the computing device.

3. The method of claim 2 , further comprising loading the 2D frames from the graphics processing unit memory to a central processing unit memory of the computing device to generate the view.

4. The method of claim 1 , further comprising rendering the multiview video for display while receiving the video stream by the computing device.

5. The method of claim 1 , further comprising receiving user input by the computing device via a user interface, the user input comprising at least one of the camera baseline and the center viewpoint.

6. The method of claim 5 , further comprising dynamically updating at least one of the camera baseline and the center viewpoint while rendering the multiview video by the computing device.

7. The method of claim 1 , wherein the view is generated by blending the subset of 2D frames.

8. The method of claim 1 ,

wherein generating the views of the multiview frame includes, for a first view:

determining time differences between the target timestamp of the first view and the 2D video timestamps adjacent to the target timestamp of the first view by the computing device; and

performing a weighted blend proportional to the time differences to generate the first view by the computing device.

9. A system configured to perform real-time multiview video conversion, the system comprising:

a central processing unit; and

a memory that stores a plurality of instructions, which, when executed, cause the central processing unit to:

load into a memory buffer a series of two-dimensional (2D) frames, each 2D frame corresponding to a respective 2D video timestamp;

determine a respective target timestamp for each view of a multiview frame based on a camera baseline and a center viewpoint, wherein the camera baseline defines an interval between views of the multiview frame, and the center viewpoint is a midpoint in a range of views of the multiview frame; and

using the determined target timestamps, generate views of the multiview frame from respective subsets of 2D frames having 2D video timestamps adjacent to the target timestamp for each respective view,

wherein the views of the multiview frame comprise part of a multiview video configured to be rendered on a multiview display.

10. The system of claim 9 , further comprising:

a graphics processing unit configured to decode a video stream comprising the series of 2D frames and load the 2D frames into a graphics processing unit memory, wherein the series of 2D frames are configured to be loaded from the graphics processing unit memory to the memory buffer.

11. The system of claim 10 , wherein the multiview video is configured to be rendered on the multiview display while decoding the video stream.

12. The system of claim 9 , wherein the instructions further cause the central processing unit to:

generate a user interface that is configured to be displayed on the multiview display; and

receive user input via the user interface, the user input comprising at least one of the camera baseline and the center viewpoint.

13. The system of claim 12 , wherein the instructions further cause the central processing unit to:

dynamically update at least one of the camera baseline and the center viewpoint while the multiview video is rendered.

14. The system of claim 9 , wherein the instructions further cause the central processing unit to:

generate a user interface that is configured to be displayed on the multiview display; and

switch between rendering the multiview video and rendering a 2D video stream in response to user input received via the user interface.

15. The system of claim 9 , wherein the views are generated by blending the subset of 2D frames.

16. The system of claim 9 , wherein the instructions further cause the central processing unit to, for a first view of the multiview frame:

determine time differences between the target timestamp of the first view and the 2D video timestamps adjacent to the target timestamp of the first view; and

perform a weighted blend proportional to the time differences to generate the first view.

17. A non-transitory, computer-readable storage medium storing executable instructions that, when executed by a processor of a computer system, performs operations of multiview video conversion in real time, the operations comprising:

receiving a video stream comprising a series of two-dimensional (2D) frames, each 2D frame corresponding to a respective 2D video timestamp;

determining a respective target timestamp for each view of a multiview frame, the target timestamps being adjacent to respective sets of 2D video timestamps that correspond to a set of 2D frames for each respective view;

blending each set of 2D frames to generate a corresponding respective view of the multiview frame; and

rendering a multiview video for display while receiving the video stream, the multiview video comprising the multiview frame.

18. The non-transitory, computer-readable storage medium of claim 17 , wherein the operations further comprise:

determining the target timestamp based on at least one of a user-specified camera baseline and a user-specified center viewpoint.

19. The non-transitory, computer-readable storage medium of claim 17 , wherein the operations further comprise:

determining time differences between a particular target timestamp and the 2D video timestamps adjacent to the particular target timestamp; and

performing a weighted blend proportional to the time differences to generate the view of the multiview frame corresponding to the particular target timestamp.

20. The non-transitory, computer-readable storage medium of claim 17 , wherein the operations further comprise:

switching between rendering the multiview video and rendering a 2D video stream in response to user input received via a user interface.

Assignments (3)

Continuity (3)

References Cited (127)