IP Library › Granted Patent US 12705836
Granted Patent B2
US 12705836 · App. 18/573,125 · Granted Aug 11, 2026

Method, apparatus, electronic device and storage medium for reconstructing 3D images

Inventors: Shaohui Jiao (Beijing, CN); Yue Wang (Beijing, CN); Dehao Zhao (Beijing, CN)
Assignee: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
G06T17/20G06T7/593G06T15/10H04N19/597G06T2207/10021G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12705836
App. No.
18/573,125
Granted
Aug 11, 2026
Kind
B2
Abstract

This disclosure discloses a method, an apparatus, an electronic device, and a storage medium for reconstructing a 3D image. The method of reconstructing a 3D image includes: receiving depth video streams of at least two camera perspectives of a same scene; determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives; performing a light field rendering on the 3D video model based on an obtained interaction parameter to obtain a plurality of target light field rendering views; and sending the plurality of target light field rendering views to a display end to construct a 3D image corresponding to the depth video streams at the display end.

Claims (55)

1 . A method of reconstructing a three-dimensional, 3D, image, comprising:

receiving depth video streams of at least two camera perspectives of a same scene;

determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives;

performing a light field rendering on the 3D video model based on an obtained interaction parameter corresponding to a display end, to obtain a plurality of target light field rendering views, wherein the plurality of target light field rendering views is represented as a two-dimensional video stream obtained after processing; and

sending the plurality of target light field rendering views to the display end to construct a 3D image corresponding to the depth video streams at the display end.

2 . The method of claim 1 , wherein the receiving depth video streams of at least two camera perspectives in a same scene comprises:

capturing depth video streams of a target area respectively based on at least two depth cameras set in the same scene.

3 . The method of claim 2 , wherein the at least two depth cameras are arranged horizontally relative to a target user in the target area.

4 . The method of claim 1 , wherein the determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives comprises:

obtaining a 3D video model corresponding to the depth video streams of the at least two camera perspectives by sequentially performing point cloud fusion, mesh generation, and texture estimation on the depth video streams of the at least two camera perspectives.

5 . The method of claim 1 , wherein the performing a light field rendering on the 3D video model based on an obtained interaction parameter, to obtain a plurality of target light field rendering views comprises:

in case where the display end is a naked-eye 3D display device, the obtained interaction parameter is a device parameter of the naked-eye 3D display device;

performing the light field rendering on the 3D video model based on the device display parameter to obtain the plurality of target light field rendering views.

6 . The method of claim 1 , wherein the performing a light field rendering on the 3D video model based on an obtained interaction parameter to obtain a plurality of target light field rendering views comprises:

in case where the display end is a non-naked eye 3D display device, the obtained interaction parameter is a device parameter of the non-naked eye 3D display device and a user head parameter corresponding to the display end;

performing the light field rendering on the 3D video model based on the device display parameter and the head parameter to obtain the plurality of target light field rendering views.

7 . The method of claim 1 , wherein the sending the plurality of target light field rendering views to a display end, to construct a 3D image corresponding to the depth video streams at the display end comprises:

compressing formats of the plurality of target light field rendering views into a 2D video format, and sending the target light field rendering views of the 2D video format to the display end, so that the display end performs a light field rearrangement on the target light field rendering views of the 2D video format based on a device parameter of the display end, to obtain the 3D image corresponding to the depth video streams.

8 . An electronic device, comprising:

at least one processor;

a storage device configured to store at least one program;

the at least one program, when executed by the at least one processor, causes the at least one processor to perform acts comprising:

receiving depth video streams of at least two camera perspectives of a same scene;

determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives;

performing a light field rendering on the 3D video model based on an obtained interaction parameter corresponding to a display end, to obtain a plurality of target light field rendering views, wherein the plurality of target light field rendering views is represented as a two-dimensional video stream obtained after processing; and

sending the plurality of target light field rendering views to the display end to construct a 3D image corresponding to the depth video streams at the display end.

9 . The electronic device of claim 8 , wherein the receiving depth video streams of at least two camera perspectives in a same scene comprises:

capturing depth video streams of a target area respectively based on at least two depth cameras set in the same scene.

10 . The electronic device of claim 9 , wherein the at least two depth cameras are arranged horizontally relative to a target user in the target area.

11 . The electronic device of claim 8 , wherein the determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives comprises:

obtaining a 3D video model corresponding to the depth video streams of the at least two camera perspectives by sequentially performing point cloud fusion, mesh generation, and texture estimation on the depth video streams of the at least two camera perspectives.

12 . The electronic device of claim 8 , wherein the performing a light field rendering on the 3D video model based on an obtained interaction parameter, to obtain a plurality of target light field rendering views comprises:

in case where the display end is a naked-eye 3D display device, the obtained interaction parameter is a device parameter of the naked-eye 3D display device;

performing the light field rendering on the 3D video model based on the device display parameter to obtain the plurality of target light field rendering views.

13 . The electronic device of claim 8 , wherein the performing a light field rendering on the 3D video model based on an obtained interaction parameter to obtain a plurality of target light field rendering views comprises:

in case where the display end is a non-naked eye 3D display device, the obtained interaction parameter is a device parameter of the non-naked eye 3D display device and a user head parameter corresponding to the display end;

performing the light field rendering on the 3D video model based on the device display parameter and the head parameter to obtain the plurality of target light field rendering views.

14 . The electronic device of claim 8 , wherein the sending the plurality of target light field rendering views to a display end, to construct a 3D image corresponding to the depth video streams at the display end comprises:

compressing formats of the plurality of target light field rendering views into a 2D video format, and sending the target light field rendering views of the 2D video format to the display end, so that the display end performs a light field rearrangement on the target light field rendering views of the 2D video format based on a device parameter of the display end, to obtain the 3D image corresponding to the depth video streams.

15 . A non-transitory storage medium containing computer-executable instructions, the computer-executable instructions, when executed by a computer processor, perform acts comprising:

receiving depth video streams of at least two camera perspectives of a same scene;

determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives;

performing a light field rendering on the 3D video model based on an obtained interaction parameter corresponding to a display end, to obtain a plurality of target light field rendering views, wherein the plurality of target light field rendering views is represented as a two-dimensional video stream obtained after processing; and

sending the plurality of target light field rendering views to the display end to construct a 3D image corresponding to the depth video streams at the display end.

16 . The storage medium of claim 15 , wherein the receiving depth video streams of at least two camera perspectives in a same scene comprises:

capturing depth video streams of a target area respectively based on at least two depth cameras set in the same scene.

17 . The storage medium of claim 16 , wherein the at least two depth cameras are arranged horizontally relative to a target user in the target area.

18 . The storage medium of claim 15 , wherein the determining a 3D video model corresponding to the depth video streams of the at least two camera perspectives comprises:

obtaining a 3D video model corresponding to the depth video streams of the at least two camera perspectives by sequentially performing point cloud fusion, mesh generation, and texture estimation on the depth video streams of the at least two camera perspectives.

19 . The storage medium of claim 15 , wherein the performing a light field rendering on the 3D video model based on an obtained interaction parameter, to obtain a plurality of target light field rendering views comprises:

in case where the display end is a naked-eye 3D display device, the obtained interaction parameter is a device parameter of the naked-eye 3D display device;

performing the light field rendering on the 3D video model based on the device display parameter to obtain the plurality of target light field rendering views.

20 . The storage medium of claim 15 , wherein the performing a light field rendering on the 3D video model based on an obtained interaction parameter to obtain a plurality of target light field rendering views comprises:

in case where the display end is a non-naked eye 3D display device, the obtained interaction parameter is a device parameter of the non-naked eye 3D display device and a user head parameter corresponding to the display end;

performing the light field rendering on the 3D video model based on the device display parameter and the head parameter to obtain the plurality of target light field rendering views.