IP Library Granted Patent US 12694623
Granted Patent B2
US 12694623 · App. 17/980,584 · Granted Jul 28, 2026

3D photos

Inventors: Alexandre Da Veiga (San Francisco, CA); Jeffrey S. Norris (Saratoga, CA); Madhurani R. Sapre (Milpitas, CA); Spencer H. Ray (San Jose, CA)
Assignee: Apple Inc.
G06T19/006G06T5/73G06T5/77G06T5/94G06T7/70G06T15/08G06T15/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12694623
App. No.
17/980,584
Granted
Jul 28, 2026
Kind
B2
Abstract

Various implementations disclosed herein include devices, systems, and methods that determine how to present a three-dimensional (3D) photo in an extended reality (XR) environment (e.g., in 3D, 2D, blurry, or not at all) based on viewing position of a user active in the XR environment relative to a placement of the 3D photo in the XR environment. In some implementations, at an electronic device having a processor, a 3D photo that is an incomplete 3D representation created based on one or more images captured by an image capture device is obtained. In some implementations, a viewing position of the electronic device relative to a placement position of the 3D photo is determined, and a presentation mode for the 3D photo is determined based on the viewing position. In some implementations, the 3D photo is provided at the placement position based on the presentation mode in the XR environment.

Claims (50)

1 . A method comprising:

at an electronic device having a processor:

obtaining a three-dimensional (3D) photo comprising an incomplete 3D representation created based on one or more images captured by an image capture device, the incomplete 3D representation missing data for providing complete views from one or more viewpoints;

determining a viewing position of the electronic device relative to a placement position of the 3D photo within an extended reality (XR) environment;

determining a presentation mode for the 3D photo based on assessing how much of a view of the 3D object is valid from the viewing position, wherein assessing how much of the view of the 3D object is valid comprises:

detecting an amount or percentage of valid pixels for the view of the 3D photo from the viewing position; and

comparing the amount or percentage of valid pixels for the view against a threshold; and

presenting the 3D photo at the placement position based on the presentation mode in the XR environment.

2 . The method of claim 1 , further comprising:

modifying the 3D photo or the presentation mode in response to the viewing position of the electronic device relative to a placement position of the 3D photo changing to a second viewing position.

3 . The method of claim 2 , wherein the second viewing position is determined to be in a first type of viewing area when at least a first threshold amount of pixels of a view of the 3D photo include image and depth values, a second type of viewing area when at least a second threshold amount and less than the first threshold amount pixels of the view of the 3D photo include the image and depth values, and a third type of viewing area otherwise.

4 . The method of claim 2 , wherein the second viewing position is determined to be in a first type viewing area or outside the first type viewing area based on movement of the image capture device during capture of the one or more images used to create the 3D photo.

5 . The method of claim 1 , wherein the presentation mode of the 3D photo is changed to a blurry presentation mode.

6 . The method of claim 1 , wherein the presentation mode of the 3D photo is changed to a 2D presentation mode.

7 . The method of claim 1 , wherein the presentation mode of the 3D photo is changed to an invisible presentation mode.

8 . The method of claim 1 , wherein lighting or shadowing of the 3D photo affects the XR environment or the lighting or the shadowing of the XR environment affects the 3D photo.

9 . The method of claim 1 , further comprising:

determining a capture relationship of the 3D photo to a ground plane using the one or more images captured by an image capture device; and

matching the ground plane capture relationship of the 3D photo to a ground plane of the XR environment.

10 . The method of claim 1 , wherein the 3D photo is a 3D cutout of the incomplete 3D representation created based on the one or more images captured by the image capture device.

11 . The method of claim 10 , wherein the 3D cutout is based on depth data, a 3D bounding region, or a 3D boundary, and wherein lighting or shadowing of the XR environment affects the 3D cutout.

12 . The method of claim 1 , wherein missing pixels in a view of the 3D photo from the viewing position are filled in based on nearby pixels with image values or depth values.

13 . The method of claim 1 , wherein the 3D photo is created using at least 1 image and depth information.

14 . The method of claim 1 , wherein the 3D photo comprises scale information that is used to scale the 3D photo in the XR environment.

15 . The method of claim 1 , wherein the 3D photo is created using a plurality of images from a single electronic device, wherein the plurality of images is from a panoramic type of single capture event.

16 . The method of claim 1 , wherein the 3D photo is created using a plurality of images from a plurality of different electronic devices, wherein the plurality of images each include at least a portion of a physical object in a physical environment.

17 . The method of claim 1 , wherein the 3D photo is created using a plurality of images from a single electronic device, wherein the plurality of images are from a plurality of different capture events that each include at least a portion of a physical object in a physical environment.

18 . The method of claim 1 , wherein the 3D photo is displayed within or on a virtual object in the XR environment.

19 . The method of claim 1 , wherein assessing how much of the view of the 3D object is valid comprises:

detecting the amount of valid pixels for the view of the 3D photo from the viewing position; and

comparing the amount of valid pixels for the view against a threshold.

20 . The method of claim 1 , wherein assessing how much of the view of the 3D object is valid comprises:

detecting the percentage of valid pixels for the view of the 3D photo from the viewing position; and

comparing the percentage of valid pixels for the view against a threshold.

21 . A system comprising:

a non-transitory computer-readable storage medium; and

one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:

obtaining a three-dimensional (3D) photo comprising an incomplete 3D representation created based on one or more images captured by an image capture device, the incomplete 3D representation missing data for providing complete views from one or more viewpoints;

determining a viewing position of the electronic device relative to a placement position of the 3D photo within an extended reality (XR) environment;

determining a presentation mode for the 3D photo based on assessing how much of a view of the 3D object is valid from the viewing position, wherein assessing how much of the view of the 3D object is valid comprises:

detecting an amount or percentage of valid pixels for the view of the 3D photo from the viewing position; and

comparing the amount or percentage of valid pixels for the view against a threshold; and

presenting the 3D photo at the placement position based on the presentation mode in the XR environment.

22 . A non-transitory computer-readable storage medium storing instructions executable via a processor to perform operations comprising:

obtaining a three-dimensional (3D) photo comprising an incomplete 3D representation created based on one or more images captured by an image capture device, the incomplete 3D representation missing data for providing complete views from one or more viewpoints;

determining a viewing position of the electronic device relative to a placement position of the 3D photo within an extended reality (XR) environment;

determining a presentation mode for the 3D photo based on assessing how much of a view of the 3D object is valid from the viewing position, wherein assessing how much of the view of the 3D object is valid comprises:

detecting an amount or percentage of valid pixels for the view of the 3D photo from the viewing position; and

comparing the amount or percentage of valid pixels for the view against a threshold; and

presenting the 3D photo at the placement position based on the presentation mode in the XR environment.