Granted Patent B2

US 12,646,189 · App. 18/235,211 · Granted Jun 2, 2026

Inventors: Mykhailo Uss (Kyiv, UA); Ruslan Iermolenko (Kyiv, UA); Olena Kolodiazhna (Kyiv, UA); Volodymyr Savin (Kyiv, UA)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G06T7/514G06T17/00G06T2207/10024G06T2207/10028

Abstract

Claims (73)

1 . A method for generating a depth map, the method comprising:

acquiring at least one image of a scene including a reflective object or a semi-transparent object;

acquiring, from the at least one image, a first depth map including a depth value for at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object;

acquiring, from the at least one image, a second depth map including a depth value for the reflective object or the semi-transparent object; and

generating a depth map for the scene based on the acquired first depth map and the acquired second depth map,

wherein the generated depth map includes the depth value for the reflective object or the semi-transparent object and the depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object.

2 . The method of claim 1 , wherein the acquiring of the second depth map comprises:

based on the scene including the reflective object and the reflective object being placed perpendicularly on an opaque surface, identifying an intersection line between the opaque surface and a surface of the reflective object;

determining a depth value of the intersection line; and

determining the depth value of the intersection line to be a depth value for the surface of the reflective object.

3 . The method of claim 1 , wherein the acquiring of the second depth map comprises:

based on the scene including the reflective object and the reflective object being placed in parallel on an opaque surface, determining a depth value of the opaque surface; and

determining the depth value of the opaque surface to be the depth value for a surface of the reflective object.

4 . The method of claim 1 , wherein the acquiring of the first depth map comprises:

generating a third depth map including no depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object;

estimating the depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object; and

synthesizing the estimated depth value for the at least one opaque object with the third depth map.

5 . The method of claim 1 , wherein the acquiring of the first depth map comprises:

predicting a depth map for the at least one opaque object from a sequence of RGB frames for the scene;

predicting the depth value for the at least one opaque object based on the reflective object and the at least one opaque object; and

synthesizing the predicted depth value for the at least one opaque object with the predicted depth map.

6 . The method of claim 1 , wherein the acquiring of the second depth map comprises:

generating a fourth depth map including no depth value for the reflective object or the semi-transparent object;

estimating the depth value for the reflective object or the semi-transparent object; and

synthesizing the estimated depth value for the reflective object or the semi-transparent object with the fourth depth map.

7 . The method of claim 1 , further comprising reconstructing a three-dimensional (3D) environment for the scene based on the generated depth map.

8 . The method of claim 1 , wherein the at least one image includes an RGB-D image.

9 . An electronic device for generating a depth map, the electronic device comprising:

a memory configured to store instructions; and

a controller coupled to the memory and configured to execute the instructions to:

acquire at least one image of a scene including a reflective object or a semi-transparent object;

acquire, from the at least one image, a first depth map including a depth value for at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object;

acquire, from the at least one image, a second depth map including a depth value for the reflective object or the semi-transparent object; and

generate a depth map for the scene based on the acquired first depth map and the acquired second depth map,

wherein the generated depth map includes the depth value for the reflective object or the semi-transparent object and the depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object.

10 . The electronic device of claim 9 , wherein the controller is further configured to execute the instructions to:

based on the scene including the reflective object and the reflective object being placed perpendicularly on an opaque surface, identify an intersection line between the opaque surface and a surface of the reflective object;

determine a depth value of the intersection line; and

determine the depth value of the intersection line to be a depth value for the surface of the reflective object.

11 . The electronic device of claim 9 , wherein the controller is further configured to execute the instructions to:

based on the scene including the reflective object and the reflective object being placed in parallel on an opaque surface, determine a depth value of the opaque surface; and

determine the depth value of the opaque surface to be the depth value for a surface of the reflective object.

12 . The electronic device of claim 9 , wherein the controller is further configured to execute the instructions to:

generate a third depth map including no depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object;

estimate the depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object; and

synthesize the estimated depth value for the at least one opaque object with the third depth map.

13 . The electronic device of claim 9 , wherein the controller is further configured to execute the instructions to:

predict a depth map for the at least one opaque object from a sequence of RGB frames for the scene;

predict the depth value for the at least one opaque object based on the reflective object and the at least one opaque object; and

synthesize the predicted depth value for the at least one opaque object with the predicted depth map.

14 . The electronic device of claim 9 , wherein the controller is further configured to execute the instructions to:

generate a fourth depth map including no depth value for the reflective object or the semi-transparent object;

estimate the depth value for the reflective object or the semi-transparent object; and

synthesize the estimated depth value for the reflective object or the semi-transparent object with the fourth depth map.

15 . The electronic device of claim 9 , wherein the controller is further configured to execute the instructions to reconstruct a three-dimensional (3D) environment for the scene based on the generated depth map.

16 . The electronic device of claim 9 , wherein the at least one image includes an RGB-D image.

17 . A non-transitory computer-readable storage medium for storing instructions for generating a depth map, wherein the instructions, when executed by a controller of an electronic device, cause the electronic device to operate a method for generating a depth map, the method comprising:

acquiring at least one image of a scene including a reflective object or a semi-transparent object;

acquiring, from the at least one image, a first depth map including a depth value for at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object;

acquiring, from the at least one image, a second depth map including a depth value for the reflective object or the semi-transparent object; and

generating a depth map for the scene based on the acquired first depth map and the acquired second depth map,

wherein the generated depth map includes the depth value for the reflective object or the semi-transparent object and the depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the acquiring of the second depth map comprises:

based on the scene including the reflective object and the reflective object being placed perpendicularly on an opaque surface, identifying an intersection line between the opaque surface and a surface of the reflective object;

determining a depth value of the intersection line; and

determining the depth value of the intersection line to be a depth value for the surface of the reflective object.

19 . The non-transitory computer-readable storage medium of claim 17 , wherein the acquiring of the second depth map comprises:

based on the scene including the reflective object and the reflective object being placed in parallel on an opaque surface, determining a depth value of the opaque surface; and

determining the depth value of the opaque surface to be the depth value for a surface of the reflective object.

20 . The non-transitory computer-readable storage medium of claim 17 , wherein the acquiring of the first depth map comprises:

generating a third depth map including no depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object;

estimating the depth value for the at least one opaque object which is reflected by the reflective object or viewed through the semi-transparent object; and

synthesizing the estimated depth value for the at least one opaque object with the third depth map.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (27)