IP Library Granted Patent US 12,452,397
Granted Patent B2
US 12,452,397 · App. 18/239,691 · Granted Oct 21, 2025

Multiview image creation system and method

Inventor: Sean Liu (San Francisco, CA)
Assignee: LEIA SPV LLC
H04N13/282H04N13/128H04N13/156H04N13/32H04N13/349H04N13/359
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,452,397
App. No.
18/239,691
Granted
Oct 21, 2025
Kind
B2
Abstract

A multiview image creation system and method create a multiview image from a single view image. Creating a multiview image includes importing a single view image and assigning the single view image to a first multiview image layer of a composite multiview image. Creating a multiview image further includes replicating the single view image into a plurality of view images of the first multiview image layer, converting a depth setting of the first multiview image layer into a plurality of shift values of corresponding view images based on an ordered number of the view images, and shifting the view images of the first multiview image layer according to the corresponding shift values. The plurality of multiview image layers may be automatically rendered in a predefined sequence as the composite multiview image on a multiview display.

Claims (46)

1. A method of creating a multiview image, the method comprising:

importing a single view image into an application, the single view image being assigned to a first multiview image layer among a plurality of multiview image layers that form a composite multiview image;

replicating the single view image into a plurality of view images of the first multiview image layer;

receiving user input comprising a depth setting for the first multiview image layer;

converting the depth setting to a plurality of shift values for corresponding view images based on an ordered number of each of the view images, the plurality of shift values being ordered such that adjacent shift values are equally spaced apart with a spacing that corresponds to the depth setting;

shifting the view images of the first multiview image layer according to the corresponding shift values in response to the first multiview image layer being selected; and

automatically rendering the plurality of multiview image layers in a predefined sequence as the composite multiview image on a multiview display.

2. The method of creating a multiview image of claim 1 , wherein the single view image is formatted in a bitmap format comprising color channel values and a transparency value for each pixel.

3. The method of creating a multiview image of claim 1 , further comprising assigning one of the view images as a base view image, wherein the shift value of the base view image is zero.

4. The method of creating a multiview image of claim 1 , wherein shifting each view image comprises shifting pixels of each view image along a horizontal direction.

5. The method of creating a multiview image of claim 1 , further comprising blending each view image of the first multiview image layer with corresponding view images of a second multiview image layer of the plurality of multiview image layers.

6. The method of creating a multiview image of claim 1 , wherein automatically rendering the plurality of multiview image layers in the predefined sequence as the composite multiview image comprises rendering an upper multiview image layer defined by the predefined sequence while omitting transparent regions of the upper multiview image layer.

7. The method of creating a multiview image of claim 1 , wherein automatically rendering the plurality of multiview image layers in the predefined sequence as the composite multiview image comprises mapping pixels of each view image to interlaced locations of the multiview display.

8. The method of creating a multiview image of claim 1 , further comprising selecting a region of the first multiview image layer and selectively shifting a portion of the view images that fall within the selected region.

9. The method of creating a multiview image of claim 1 , further comprising formatting the composite multiview image in a user-specified multiview file format.

10. A multiview image generation system comprising:

a multiview display;

a processor, and

a memory that stores a plurality of instructions, which, when executed, cause the processor to:

select a first multiview image layer among a plurality of multiview image layers that form a composite multiview image, the first multiview image layer comprising a plurality of view images;

receive user input comprising a depth setting for the first multiview image layer;

convert the depth setting to a plurality of shift values for corresponding view images based on an ordered number of each of the view images, the plurality of shift values being ordered such that adjacent shift values are equally spaced apart with a spacing that corresponds to the depth setting;

shift the view images of the selected first multiview image layer according to the corresponding shift values; and

automatically render the plurality of multiview image layers in a predefined sequence as the composite multiview image on the multiview display,

wherein the first multiview image layer is a newly created multiview image layer, and

wherein each ordered number corresponds to a relative position of each of the view images.

11. The multiview image generation system of claim 10 , wherein the plurality of the view images are identical view images that are replicated from a single view image.

12. The multiview image generation system of claim 10 , wherein one of the view images is assigned as a base view image, wherein the shift value of the base view image is zero.

13. The multiview image generation system of claim 10 , wherein the plurality of instructions, which, when executed, further cause the processor to:

blend each view image of the first multiview image layer with corresponding view images of a second multiview image layer of the plurality of multiview image layers.

14. The multiview image generation system of claim 10 , further comprising a shader configured to iteratively render the plurality of multiview image layers in the predefined sequence by mapping pixels of each multiview image layer to interlaced locations of the multiview display.

15. The multiview image generation system of claim 10 , wherein the multiview display is configured to provide broad-angle emitted light during a two-dimensional (2D) mode using a broad-angle backlight;

wherein the multiview display is configured to provide directional emitted light during a multiview mode using a multiview backlight having an array of multibeam elements, the directional emitted light comprising a plurality of directional light beams provided by each multibeam element of the multibeam element array;

wherein the multiview display is configured to time multiplex the 2D and multiview modes using a mode controller to sequentially activate the broad-angle backlight during a first sequential time interval corresponding to the 2D mode and the multiview backlight during a second sequential time interval corresponding to the multiview mode; and

wherein directions of directional light beams of the directional light beam plurality correspond to different view directions of the composite multiview image.

16. The multiview image generation system of claim 15 , wherein the multiview display is configured to guide light in a light guide as guided light; and

wherein the multiview display is configured to scatter out a portion of the guided light as the directional emitted light using multibeam elements of the multibeam element array, each multibeam element of the multibeam element array comprising one or more of a diffraction grating, a micro-refractive element, and a micro-reflective element.

17. A non-transitory, computer-readable storage medium storing executable instructions that, when executed by a processor of a computing system, implement operations to create a multiview image from a depth map. the operations comprising:

assigning a single view image to a first multiview image layer;

loading the depth map for the first multiview image layer;

modifying one or both of a size and a shape of the depth map to provide a modified depth map;

synthesizing a plurality of view images from the single view image and from the modified depth map for the first multiview image layer, the plurality of view images being ordered such that adjacent view images are equally spaced apart with a spacing that corresponds to a property of the modified depth map; and

automatically rendering the first multiview image layer and a second multiview image layer in a predefined sequence as a composite multiview image on a multiview display.

18. The non-transitory, computer-readable storage medium of claim 17 , wherein synthesizing a plurality of view images comprises predicting the plurality of view images using a neural network.

19. The non-transitory, computer-readable storage medium of claim 17 , wherein automatically rendering comprises mapping pixels of each view image to interlaced locations of the multiview display.

20. The non-transitory, computer-readable storage medium of claim 17 , wherein a user interface is configured to overlay the depth map on the single view image and receive user input to modify one or both of a size and a shape of the depth map.

Assignments (3)
SECURITY INTEREST Recorded Nov 4, 2024
From: LEIA, INC.; LEIA SPV LLC; DIMENCO HOLDING B.V.
To: LELIS, INC., AS AGENT
Reel/Frame 069296/0265 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 29, 2023
From: LEIA INC.
To: LEIA SPV LLC
Reel/Frame 065984/0341 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2023
From: LIU, SEAN
To: LEIA INC.
Reel/Frame 064743/0153 →
Continuity (2)
Continuation PCTUS2021025687 · Apr 4, 2021
Related Publication 20230412789A1 · Dec 21, 2023
References Cited (60)
US 6417850B1 · Kang · 2002 [cited by examiner]
US 8146277B2 · Engel · 2012 [cited by applicant]
US 8970582B2 · An et al. · 2015 [cited by applicant]
US 9123176B2 · Lu et al. · 2015 [cited by applicant]
US 20120170833A1 · Kokojima et al. · 2012 [cited by applicant]
US 20130027389A1 · Wang · 2013 [cited by applicant]
US 20140049611A1 · Woo · 2014 [cited by examiner]
US 20150130793A1 · Han · 2015 [cited by examiner]
US 20160014400A1 · Han · 2016 [cited by examiner]
US 20190052864A1 · Xu · 2019 [cited by examiner]
US 20200160055A1 · Nakamura · 2020 [cited by examiner]
US 20230030728A1 · Dahlquist · 2023 [cited by applicant]
CA 3213167 · 2022 [cited by applicant]
CN 111127538A · 2020 [cited by applicant]
JP 2001167289 · 2001 [cited by applicant]
JP 2013005376 · 2013 [cited by applicant]
JP 2013017182 · 2013 [cited by applicant]
JP 2013134787 · 2013 [cited by applicant]
JP 2015022458 · 2015 [cited by applicant]
JP 2021530181 · 2021 [cited by applicant]
JP 2022530212 · 2022 [cited by applicant]
JP 2024514534 · 2024 [cited by applicant]
KR 20140033531A · 2014 [cited by applicant]
KR 20150055322A · 2015 [cited by applicant]
KR 20230145421 · 2023 [cited by applicant]
TW 201218741A · 2012 [cited by applicant]
TW 201316751 · 2013 [cited by applicant]
TW 201401224 · 2014 [cited by applicant]
TW 202304199 · 2023 [cited by applicant]
TW 1823323 · 2023 [cited by applicant]
WO 2004036286 · 2004 [cited by applicant]
WO 2020010183 · 2020 [cited by applicant]
WO 2020219400 · 2020 [cited by applicant]
WO 2022108609A1 · 2022 [cited by applicant]
WO 2022125128A1 · 2022 [cited by applicant]
WO 2022173611A1 · 2022 [cited by applicant]
WO 2022182368A1 · 2022 [cited by applicant]
WO 2022182369A1 · 2022 [cited by applicant]
WO 2022182376A1 · 2022 [cited by applicant]
WO 2023014576A1 · 2023 [cited by applicant]
WO 2023129214A1 · 2023 [cited by applicant]
International Search Report and Written Opinion (ISRWO) by International Searching Authority (ISA) Korean Intellectual Property Office (KIPO) dated Dec. 24, 2021 (10 pages) for foreign counterpart parent International A… [cited by applicant]
Olivia Wiles, et al., “SynSin: End-to-end View Synthesis from a Single Image,” arXiv: 1912.08804v2 [cs.CV], Apr. 18, 2020, pp. 1-22, from: <https://arxiv.org/abs/1912.08804>. [cited by applicant]
Tinghui Zhou et al., “Stereo Magnification: Learning view synthesis using multiplane images,” arXiv: 1805.09817v1 [cs.CV], May 24, 2018, pp. 1-12, from: <https://arxiv.org/abs/1805.09817>. [cited by applicant]
Richard Tucker et al., “Single-View View Synthesis with Multiplane Images,” arXiv: 2004.11364v1 [cs.CV], Apr. 23, 2020, pp. 1-10, from: <https://arxiv.org/abs/2004.11364>. [cited by applicant]
Wen-Nung Lie, “Chapter 5: View synthesis techniques,” National Chung Cheng University (CCU), Taiwan, 2011, 51 pages, Internet document. [cited by applicant]
“Canadian Application Serial No. 3,213, 167, Voluntary Amendment filed Sep. 11, 2023”, 51 pgs. [cited by applicant]
“Japanese Application Serial No. 2023-560964, Amendment filed Dec. 18, 2023”, W English Claims, 27 pgs. [cited by applicant]
“Taiwanese Application Serial No. 111112833, Amendment filed Apr. 1, 2022”, W English Claims, 55 pgs. [cited by applicant]
“Taiwanese Application Serial No. 111112833, Office Action mailed Mar. 15, 2023”, W English Translation, 18 pgs. [cited by applicant]
“Taiwanese Application Serial No. 111112833, Response filed Jun. 12, 2023 to Office Action mailed Mar. 15, 2023”, W O English Claims, 16 pgs. [cited by applicant]
“European Application Serial No. 21936198.7, Extended European Search Report mailed Dec. 16, 2024”, 8 pgs. [cited by applicant]
“Canadian Application Serial No. 3,213,167, Office Action mailed Dec. 13, 2024”, 3 pgs. [cited by applicant]
“Japanese Application Serial No. 2023-560964, Notification of Reasons for Rejection mailed Feb. 4, 2025”, W English Translation, 14 pgs. [cited by applicant]
“Toshibas Technological Achievements in 2013”, w English Translation, (Mar. 1, 2014), 5 pgs. [cited by applicant]
“Canadian Application Serial No. 3,213,167, Response filed to Apr. 14, 2025 Office Action mailed Dec. 13, 2024”, 9 pgs. [cited by applicant]
Rehg, J M, “markVideo editing using figure tracking and image mark-markbased rendering mark”, Image Processing, Proceedings. International Conference on Sep. 10-13, 2000, vol. 2,, [Online] Retrieved from the internet:ht… [cited by applicant]
“European Application Serial No. 21936198.7, Response filed Jun. 25, 2025 to Extended European Search Report mailed Dec. 16, 2024”, 19 pgs. [cited by applicant]
“Japanese Application Serial No. 2023-560964, Notification of Reasons for Refusal mailed Jul. 8, 2025”, w English Translation, 10 pgs. [cited by applicant]
“Korean Application Serial No. 10-2023-7031127, Notice of Preliminary Rejection mailed Jul. 3, 2025”, w English Translation, 13 pgs. [cited by applicant]