IP Library Granted Patent US 12,316,821
Granted Patent B2
US 12,316,821 · App. 18/122,037 · Granted May 27, 2025

Multiview display system and method with adaptive background

Inventors: Daniel Geisler (Salida, CA); David A. Fattal (Mountain View, CA)
Assignee: LEIA Inc.
H04N13/125G06T5/50G06T5/70H04N13/15H04N13/275H04N13/282G06T2207/20212
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,316,821
App. No.
18/122,037
Granted
May 27, 2025
Kind
B2
Abstract

An adaptive background multiview image display system and method provides improved multiview image quality. Systems and methods may involve generating crosstalk data that reduces crosstalk between a first view of subject image and a second view of the subject image. The subject image may be a multiview image to be overlaid on a background image. A crosstalk violation may be detected in the subject image based on the crosstalk data. At least one of a color value or a brightness value of the background image is determined according to a degree of the crosstalk violation to generate the background image. The subject image may then be overlaid on the generated background image.

Claims (36)

1. A computer-implemented method of adapting a background of a multiview image, the method comprising:

generating crosstalk data to reduce crosstalk between a first view of a subject image and a second view of the subject image, the subject image comprising a multiview image to be overlaid on a background image;

identifying a crosstalk violation in the crosstalk data, wherein the crosstalk violation is determined based on a pixel value of the subject image falling below a specified threshold pixel value;

modifying a visual parameter of the background image based on at least one of a location or degree of the crosstalk violation to compensate for removing the crosstalk violation in the subject image; and

overlaying the subject image on the background image, the subject image and background image being rendered on a multiview display.

2. The method of adapting a background of a multiview image of claim 1 , wherein generating the crosstalk data comprises separately generating the crosstalk data for each color channel of the first view of the subject image and the second view of the subject image.

3. The method of adapting a background of a multiview image of claim 1 , wherein the crosstalk violation is determined based on a negative pixel value of the subject image.

4. The method of adapting a background of a multiview image of claim 1 , wherein modifying the visual parameter comprises increasing at least one of a color value or a brightness value of the background image based on the location of the crosstalk violation.

5. The method of adapting a background of a multiview image of claim 1 , wherein the background image comprises at least one of a halo or color gradient positioned according to the location of the crosstalk violation.

6. The method of adapting a background of a multiview image of claim 1 , wherein the crosstalk violation is removed from the subject image by setting pixel values corresponding to the crosstalk violation to zero.

7. The method of adapting a background of a multiview image of claim 1 , further comprising identifying the location of the crosstalk violation by detecting an edge formed by a set of pixels that correspond to pixel values associated with the crosstalk violation.

8. A multiview image display system employing an adaptive background, the system comprising:

a processor; and

a memory that stores a plurality of instructions, which, when executed, cause the processor to:

receive a subject image configured to be overlaid on a background image, the subject image being a multiview image;

execute a crosstalk cancellation operation to generate crosstalk data based on a first view of the subject image and a second view of the subject image;

identify a portion of the subject image associated with a crosstalk violation based on the crosstalk data, wherein the crosstalk violation is based on a pixel value of the subject image falling below a specified threshold pixel value; and

modify a visual parameter of the background image at a location corresponding to the identified portion of the subject image, the subject image and background image configured to be rendered on a multiview display.

9. The multiview image display system employing an adaptive background of claim 8 , wherein the crosstalk violation is determined based on a negative pixel value in the subject image.

10. The multiview image display system employing an adaptive background of claim 8 , wherein the subject image is formattable in format defined by a plurality of color channels.

11. The multiview image display system employing an adaptive background of claim 10 , wherein the crosstalk data is generated separately for each color channel among the plurality of color channels.

12. The multiview image display system employing an adaptive background of claim 8 , wherein the visual parameter comprises at least one of a color pixel value or a brightness pixel value.

13. The multiview image display system employing an adaptive background of claim 8 , wherein the plurality of instructions, which, when executed, further cause the processor to determine the visual parameter to generate at least one of a halo effect or color gradient based on identifying a location of the portion of the subject image and mapping the location to the location on the background image.

14. The multiview image display system employing an adaptive background of claim 8 , wherein the crosstalk violation is removed from the subject image by setting a pixel value corresponding to the crosstalk violation to zero, the visual parameter being modified to compensate for removing the crosstalk violation.

15. The multiview image display system employing an adaptive background of claim 8 , wherein the plurality of instructions, which, when executed, further cause the processor to identify a location of the portion by identifying a group of pixels having an average pixel value that falls below a threshold pixel value.

16. A non-transitory, computer-readable storage medium storing executable instructions that, when executed by a processor of a computer system, implement adapting a background image comprising:

generating crosstalk data to reduce crosstalk between a first view of subject image and a second view of the subject image, the subject image comprising a multiview image;

detecting a crosstalk violation in the subject image based on the crosstalk data, wherein the crosstalk violation is based on a pixel value of the subject image falling below a specified threshold pixel value;

modifying at a pixel value of the background image according to a degree of the crosstalk violation; and

overlaying the subject image on the background image, the subject image and background image configured to be rendered on a multiview display.

17. The non-transitory, computer-readable storage medium of claim 16 , wherein the executable instructions that, when executed by the processor of the computer system, further implement adapting the background image comprising:

determining a location of a portion of the subject image that corresponds to the crosstalk violation; and

setting the pixel value of the background image according to the determined location.

18. The non-transitory, computer-readable storage medium of claim 17 , wherein the pixel value of the background image comprises a pixel values of different pixels that are set to form a halo positioned according to the location of the portion.

19. The non-transitory, computer-readable storage medium of claim 17 , wherein the pixel value of the background image comprises a plurality of different pixel values that are set to form a color gradient based on the location of the portion.

20. The non-transitory, computer-readable storage medium of claim 16 , wherein generating the crosstalk data comprises separately generating crosstalk data for each color channel of the first view of the subject image and the second view of the subject image.

Assignments (3)
SECURITY INTEREST Recorded Nov 4, 2024
From: LEIA, INC.; LEIA SPV LLC; DIMENCO HOLDING B.V.
To: LELIS, INC., AS AGENT
Reel/Frame 069296/0265 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 29, 2023
From: LEIA INC.
To: LEIA SPV LLC
Reel/Frame 065984/0341 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 18, 2023
From: GEISLER, DANIEL; FATTAL, DAVID A.
To: LEIA INC.
Reel/Frame 063026/0986 →
Continuity (3)
Continuation PCTUS2020055946 · Oct 16, 2020
Provisional Application 63081274 · Sep 21, 2020
Related Publication 20230217000A1 · Jul 6, 2023
References Cited (81)
US 6532008B1 · Guralnick · 2003 [cited by examiner]
US 9128226B2 · Fattal et al. · 2015 [cited by applicant]
US 9201270B2 · Fattal et al. · 2015 [cited by applicant]
US 9298168B2 · Taff et al. · 2016 [cited by applicant]
US 9389415B2 · Fattal et al. · 2016 [cited by applicant]
US 9459461B2 · Santori et al. · 2016 [cited by applicant]
US 9557466B2 · Fattal · 2017 [cited by applicant]
US 9785119B2 · Taff et al. · 2017 [cited by applicant]
US 10345505B2 · Fattal · 2019 [cited by applicant]
US 10551546B2 · Fattal · 2020 [cited by applicant]
US 10649128B2 · Fattal et al. · 2020 [cited by applicant]
US 10798371B2 · Fattal · 2020 [cited by applicant]
US 10802212B2 · Fattal · 2020 [cited by applicant]
US 10830939B2 · Fattal et al. · 2020 [cited by applicant]
US 10884175B2 · Fattal · 2021 [cited by applicant]
US 10928677B2 · Aieta et al. · 2021 [cited by applicant]
US 11016235B2 · Fattal et al. · 2021 [cited by applicant]
US 11048036B2 · Ma et al. · 2021 [cited by applicant]
US 20030117489A1 · Jones et al. · 2003 [cited by applicant]
US 20070228927A1 · Kindler et al. · 2007 [cited by applicant]
US 20080036696A1 · Slavenburg et al. · 2008 [cited by applicant]
US 20080246759A1 · Summers · 2008 [cited by applicant]
US 20090244266A1 · Brigham · 2009 [cited by applicant]
US 20090322986A1 · Wei et al. · 2009 [cited by applicant]
US 20100134493A1 · Kwak et al. · 2010 [cited by applicant]
US 20110279451A1 · Haga · 2011 [cited by applicant]
US 20120062709A1 · Kerofsky et al. · 2012 [cited by applicant]
US 20120257018A1 · Shigemura et al. · 2012 [cited by applicant]
US 20130135500A1 · Theuwissen · 2013 [cited by applicant]
US 20130169518A1 · Wu et al. · 2013 [cited by applicant]
US 20140022340A1 · Dane et al. · 2014 [cited by applicant]
US 20140085432A1 · Wu et al. · 2014 [cited by applicant]
US 20140192170A1 · Samadani et al. · 2014 [cited by applicant]
US 20160044305A1 · Kim · 2016 [cited by examiner]
US 20160134859A1 · Tang · 2016 [cited by applicant]
US 20170111633A1 · Kang · 2017 [cited by examiner]
US 20170363794A1 · Wan et al. · 2017 [cited by applicant]
US 20180252931A1 · Kim-Whitty · 2018 [cited by applicant]
US 20210364816A1 · Fattal et al. · 2021 [cited by applicant]
US 20210368158A1 · Fattal et al. · 2021 [cited by applicant]
US 20220075183A1 · Lee · 2022 [cited by examiner]
CA 3193491 · 2022 [cited by applicant]
CN 101123736 · 2008 [cited by applicant]
CN 105376558 · 2016 [cited by applicant]
CN 116195246 · 2023 [cited by applicant]
EP 4214924 · 2023 [cited by applicant]
JP 2000004455A · 2000 [cited by applicant]
JP 2009239596 · 2019 [cited by applicant]
JP 2023545634 · 2023 [cited by applicant]
KR 20080089649 · 2008 [cited by applicant]
KR 20140051333A · 2014 [cited by applicant]
KR 20230061492 · 2023 [cited by applicant]
TW 202220439 · 2022 [cited by applicant]
TW 1830056 · 2024 [cited by applicant]
WO 2012038856A1 · 2012 [cited by applicant]
WO 2012150033 · 2012 [cited by applicant]
WO 2022060387 · 2022 [cited by applicant]
International Search Report and Written Opinion (ISRWO) by International Searching Authority (ISA) Korean Intellectual Property Office (KIPO) dated Jun. 18, 2021 (10 pages) for foreign counterpart parent International A… [cited by applicant]
Fattal, David et al., “A multi-directional backlight for a wide-angle, glasses-free three-dimensional display,” Nature, Mar. 21, 2013, pp. 348-351, vol. 495, Macmillan Publishers Limited, 2013. [cited by applicant]
Kee, Edwin., “Hitachi Full Parallax 3D Display Offers Mind Bending Visuals,” http://www.ubergizmo.com/2011/10/hitachi-full-parallax-3d-display-offers-mind-bending-visuals, Oct. 4, 2011, 2 pages. [cited by applicant]
Reichelt et al., “Holographic 3-D Displays—Electro-holography within the Grasp of Commercialization,” Advances In Lasers and Electro-Optics, Optics, Nelson Costa and Adolfo Cartaxo (Ed.), (2010), pp. 683-711, ISBN: 978-… [cited by applicant]
Travis et al., “Collimated light from a waveguide for a display backlight,” Optics Express, Oct. 2009, pp. 19714-19719, vol. 17, No. 22. [cited by applicant]
Xu et al., “Computer-Generated Holography for Dynamic Display of 3D Objects with Full Parallax,” International Journal of Virtual Reality, 2009, pp. 33-38, vol. 8, No. 2. [cited by applicant]
Son, Jung-Young et al., “Three-Dimensional Imaging Methods Based on Multiview Images,” IEEE/OSA Journal of Display Technology, Sep. 2005, pp. 125-140, vol. 1, No. 1. [cited by applicant]
“Korean Application Serial No. 10-2023-7011296, Notice of Preliminary Rejection mailed Mar. 26, 2024”, w English Translation, 10 pgs. [cited by applicant]
“International Application Serial No. PCT US2020 055946, International Search Report mailed Jun. 18, 2021”, 3 pgs. [cited by applicant]
“International Application Serial No. PCT US2020 055946, Written Opinion mailed Jun. 18, 2021”, 5 pgs. [cited by applicant]
“International Application Serial No. PCT US2020 055946, International Preliminary Report on Patentability mailed Mar. 30, 2023”, 7 pgs. [cited by applicant]
“European Application Serial No. 20954310.7, Response filed Oct. 24, 2023 to Communication pursuant to Rules 161(1) and 162 EPC filed May 2, 2023”, 21 pgs. [cited by applicant]
“Taiwanese Application Serial No. 110131899, Office Action mailed Mar. 17, 2022”, w English translation, 15 pgs. [cited by applicant]
“Taiwanese Application Serial No. 110131899, Response filed May 25, 2022 to Office Action mailed Mar. 17, 2022”, 3 pgs. [cited by applicant]
“Taiwanese Application Serial No. 110131899, Office Action mailed Jun. 14, 2023”, w English translation, 28 pgs. [cited by applicant]
“Taiwanese Application Serial No. 110131899, Response filed Sep. 14, 2023 to Office Action mailed Jun. 14, 2023”, w English translation of claims, 16 pgs. [cited by applicant]
“Canadian Application Serial No. 3,193,491, Examiners Rule 86(2) Report mailed Jul. 23, 2024”, 5 pgs. [cited by applicant]
“Canadian Application Serial No. 3,193,491, Response filed Nov. 19, 2024 to Examiners Rule (86(2) Report mailed Jul. 23, 2024”, 4 pgs. [cited by applicant]
“European Application Serial No. 20954310.7, Extended European Search Report mailed Jul. 12, 2024”, 8 pgs. [cited by applicant]
“European Application Serial No. 20954310.7, Response filed Oct. 30, 2024 to Extended European Search Report mailed Jul. 12, 2024”, 19 pgs. [cited by applicant]
“Japanese Application Serial No. 2023-518114, Notification of Reasons for Refusal mailed Jul. 30, 2024”, w/ English translation, 11 pgs. [cited by applicant]
“Korean Application Serial No. 10-2023-7011296, Response filed Jul. 17, 2024 to Notice of Preliminary Rejection mailed Mar. 26, 2024”, w/ English claims, 19 pgs. [cited by applicant]
“Chinese Application Serial No. 202080105391.7, Office Action mailed Feb. 19, 25”, w English Translation, 12 pgs. [cited by applicant]
“Japanese Application Serial No. 2023-518114, Examiners Decision of Final Refusal mailed Apr. 1, 25”, W English Translation, 11 pgs. [cited by applicant]