Granted Patent B2

US 12,646,246 · App. 18/443,936 · Granted Jun 2, 2026

Inventor: Joon Soo Kim (Daejeon, KR)

Assignee: Electronics and Telecommunications Research Institute

G06T15/506G06T7/11G06T7/73G06T2207/20081G06T2207/20084

Abstract

Claims (57)

1 . A method performed by a device to reconstruct a three-dimensional (3D) shape based on multiple light sources, the method includes:

obtaining a set of measured images of a specimen by photographing the specimen under different lighting conditions;

defining a first three-dimensional grid within a 3D reconstruction area including all or part of the specimen;

obtaining a first set of coordinate values for calculating a scattering potential value within a first 3D grid or a frequency grid region corresponding to the first 3D grid;

inputting the first set of coordinate values into an artificial neural network model to obtain an inference result related to a scattered potential;

obtaining a set of predicted images corresponding to the different lighting conditions based on the inference result related to the scattering potential;

training the artificial neural network model based on differences between the set of predicted images and the set of the measured images;

obtaining a second set of coordinate values including at least one coordinate of a second 3D grid in an area for performing final 3D reconstruction;

inputting the second set of coordinate values into the artificial neural network model to obtain output data related to scattering potential; and

obtaining a 3D refractive index distribution of the specimen based on the output data,

wherein a domain type related to the artificial neutral network model is determined based on a scattering characteristic of a sample corresponding to the set of measured images,

wherein based on a domain type being determined to be a spatial domain: the inference result includes scattering potential data, and the set of predicted images is obtained based on scattering potential spectrum data obtained from the scattering potential data, and

wherein based on the domain type being determined to be a frequency domain: the inference result includes the scattering potential spectrum data, and the set of predicted images is obtained based on the scattering potential spectrum data.

2 . The method of claim 1 , wherein:

the multiple light sources include a plurality of light emitting diodes (LEDs) configuring the different lighting conditions,

the set of measured images is obtained through light emitted through each of the plurality of LEDs, and

an angle of incidence of the each of the plurality of LEDs is different from each other.

3 . The method of claim 1 , wherein:

the training the artificial neural network model includes updating a weight of the artificial neural network model through gradient descent using a loss function calculated based on difference between the set of predicted images and the set of measured images.

4 . A device that reconstructs 3-dimensional (3D) shape based on multiple light sources, the device comprising:

at least one memory; and

at least one processor,

wherein the at least one processor is configured to:

obtain a set of measured images of a specimen by photographing the specimen under different lighting conditions;

define a first three-dimensional grid within a 3D reconstruction area including all or part of the specimen;

obtain a first set of coordinate values for calculating a scattering potential value within a first 3D grid or a frequency grid region corresponding to the first 3D grid;

input the first set of coordinate values into an artificial neural network model to obtain an inference result related to a scattered potential;

obtain a set of predicted images corresponding to the different lighting conditions based on the inference result related to the scattering potential;

train the artificial neural network model based on differences between the set of predicted images and the set of the measured images;

obtain a second set of coordinate values including at least one coordinate of a second 3D grid in an area for performing final 3D reconstruction;

input the second set of coordinate values into the artificial neural network model to obtain output data related to scattering potential; and

obtain a 3D refractive index distribution of the specimen based on the output data,

wherein a domain type related to the artificial neutral network model is determined based on a scattering characteristic of a sample corresponding to the set of measured images,

wherein based on a domain type being determined to be a spatial domain: the inference result includes scattering potential data, and the set of predicted images is obtained based on scattering potential spectrum data obtained from the scattering potential data, and

wherein based on the domain type being determined to be a frequency domain: the inference result includes the scattering potential spectrum data, and the set of predicted images is obtained based on the scattering potential spectrum data.

5 . The device of claim 4 , wherein:

the multiple light sources include a plurality of light emitting diodes (LEDs) configuring the different lighting conditions,

the set of measured images is obtained through light emitted through each of the plurality of LEDs, and

an angle of incidence of the each of the plurality of LEDs is different from each other.

6 . The device of claim 4 ,

wherein the at least one processor is configured to update a weight of the artificial neural network model through gradient descent using a loss function calculated based on difference between the set of predicted images and the set of measured images.

7 . A system for reconstructing a 3-dimensional (3D) shape, the system comprising:

a device for restoring 3D shape; and

multiple light source system that obtains measured image data through multiple light sources;

the device is configured to:

obtain a set of measured images of a specimen by photographing the specimen under different lighting conditions;

define a first three-dimensional grid within a 3D reconstruction area including all or part of the specimen;

obtain a first set of coordinate values for calculating a scattering potential value within a first 3D grid or a frequency grid region corresponding to the first 3D grid;

input the first set of coordinate values into an artificial neural network model to obtain an inference result related to a scattered potential;

obtain a set of predicted images corresponding to the different lighting conditions based on the inference result related to the scattering potential;

train the artificial neural network model based on differences between the set of predicted images and the set of the measured images;

obtain a second set of coordinate values including at least one coordinate of a second 3D grid in an area for performing final 3D reconstruction;

input the second set of coordinate values into the artificial neural network model to obtain output data related to scattering potential; and

obtain a 3D refractive index distribution of the specimen based on the output data,

wherein a domain type related to the artificial neutral network model is determined based on a scattering characteristic of a sample corresponding to the set of measured images,

wherein based on a domain type being determined to be a spatial domain: the inference result includes scattering potential data, and the set of predicted images is obtained based on scattering potential spectrum data obtained from the scattering potential data, and

wherein based on the domain type being determined to be a frequency domain: the inference result includes the scattering potential spectrum data, and the set of predicted images is obtained based on the scattering potential spectrum data.

Assignments (1)

Priority Claims (2)

Continuity (1)

References Cited (21)