IP Library Patent Application 16525471
Patent Application
App. No. 16/525,471

METHOD AND SYSTEM FOR GENERATING 3D IMAGE FROM 2D IMAGE

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
16/525,471
Abstract

Systems and methods for generating and displaying a three dimensional (3D) images are described. The systems and methods may provide for receiving a two dimensional (2D) photo image and a depth map, the depth map comprising a 2D depth image including a plurality of pixels, each pixel representing a depth value for a corresponding pixel of the 2D photo image, constructing a 3D mesh from the depth map, the 3D mesh comprising a 3D representation of the depth map and comprising a plurality of vertices and edges, mapping the 2D photo image as a texture on the 3D mesh to create a 3D image, interpolating a plurality of missing pixel values in the texture from adjacent pixel value, and rendering the 3D image.

Claims (60)

1 . A computer-implemented method for generating and displaying a three dimensional (3D) image, the method comprising:

receiving a two dimensional (2D) photo image and a depth map, the depth map comprising a 2D depth image including a plurality of pixels, each pixel representing a depth value for a corresponding pixel of the 2D photo image;

constructing a 3D mesh from the depth map, the 3D mesh comprising a 3D representation of the depth map and comprising a plurality of vertices and edges;

mapping the 2D photo image as a texture on the 3D mesh to create a 3D image;

interpolating a plurality of missing pixel values in the texture from adjacent pixel value; and

rendering the 3D image.

2 . The computer-implemented method of claim 1 , further comprising:

receiving the depth map as an uploaded file from a user.

3 . The computer-implemented method of claim 1 , further comprising:

generating, by a machine learning depth prediction model, the depth map from the 2D photo image, wherein the machine learning depth prediction model is trained on a dataset of images and corresponding depth maps.

4 . The computer-implemented method of claim 3 , further comprising:

displaying a visual element, the visual element including a portion of the 2D photo image and a portion of the depth map; and

displaying a slider on top of the visual element, wherein the slider is configured to be adjustable and allow increasing and decreasing the area of the visual element comprising the 2D photo image and the depth map.

5 . The computer-implemented method of claim 1 , wherein:

the depth map is generated simultaneously with the 2D photo image by a camera with a depth sensing system.

6 . The computer-implemented method of claim 1 , further comprising:

displaying the 3D image in an online banner advertisement.

7 . The computer-implemented method of claim 6 , further comprising:

receiving user input in a web browser and changing a user viewpoint for viewing the 3D image in response to the user input;

displaying the 3D image using HTML5 and Javascript.

8 . The computer-implemented method of claim 7 , further comprising:

transmitting the online banner advertisement to an online advertising network; and

receiving payment from the online advertising network per click on the online banner advertisement.

9 . The computer-implemented method of claim 1 , further comprising:

receiving the 2D photo image as an uploaded file from a user;

receiving the depth map as an uploaded file from the user;

receiving a selection of a banner ad format by the user; and

receiving a user input to initiate generation of the 3D image.

10 . The computer-implemented method of claim 1 , further comprising:

displaying a 2D overlay on top of the 3D image, the 2D overlay including one or more interactive use interface elements.

11 . An apparatus for generating and displaying a three dimensional (3D) images, comprising: a processor and a memory storing instructions and in electronic communication with the processor, the processor being configured to execute the instructions to:

receive a two dimensional (2D) photo image and a depth map, the depth map comprising a 2D depth image including a plurality of pixels, each pixel representing a depth value for a corresponding pixel of the 2D photo image;

construct a 3D mesh from the depth map, the 3D mesh comprising a 3D representation of the depth map and comprising a plurality of vertices and edges;

map the 2D photo image as a texture on the 3D mesh to create a 3D image;

interpolate a plurality of missing pixel values in the texture from adjacent pixel value; and

render the 3D image.

12 . The apparatus of claim 11 , the processor being further configured to execute the instructions to:

receive the depth map as an uploaded file from a user.

13 . The apparatus of claim 11 , the processor being further configured to execute the instructions to:

generate, by a machine learning depth prediction model, the depth map from the 2D photo image, wherein the machine learning depth prediction model is trained on a dataset of images and corresponding depth maps.

14 . The apparatus of claim 13 , the processor being further configured to execute the instructions to:

display a visual element, the visual element including a portion of the 2D photo image and a portion of the depth map; and

display a slider on top of the visual element, wherein the slider is configured to be adjustable and allow increasing and decreasing the area of the visual element comprising the 2D photo image and the depth map.

15 . The apparatus of claim 11 , wherein:

the depth map is generated simultaneously with the 2D photo image by a camera with a depth sensing system.

16 . The apparatus of claim 11 , the processor being further configured to execute the instructions to:

display the 3D image in an online banner advertisement.

17 . The apparatus of claim 16 , the processor being further configured to execute the instructions to:

receive user input in a web browser and changing a user viewpoint for viewing the 3D image in response to the user input;

display the 3D image using HTML5 and Javascript.

18 . The apparatus of claim 17 , the processor being further configured to execute the instructions to:

transmit the online banner advertisement to an online advertising network; and

receive payment from the online advertising network per click on the online banner advertisement.

19 . The apparatus of claim 11 , the processor being further configured to execute the instructions to:

receive the 2D photo image as an uploaded file from a user;

receive the depth map as an uploaded file from the user;

receive a selection of a banner ad format by the user; and

receive a user input to initiate generation of the 3D image.

20 . The apparatus of claim 11 , the processor being further configured to execute the instructions to:

display a 2D overlay on top of the 3D image, the 2D overlay including one or more interactive use interface elements.

Assignments (2)
CHANGE OF NAME Recorded Dec 30, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058600/0190 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2019
From: PHAISANWIPHATPONG, PRACHAYA; RUCKER, MICHAEL; PHANVILAI, SITTIPHOL
To: SOCIAL NATION, INC.
Reel/Frame 049904/0059 →