Three-dimensional reconstruction method and system, and storage medium

1 . A method for training a three-dimensional reconstruction model, comprising:

generating an initial voxel envelope of a target object based on an image that is obtained by photographing the target object at a plurality of angles of view;

performing random sampling on points inside the initial voxel envelope, so as to obtain a set of sampling points;

performing global feature extraction on the image, so as to obtain a global feature map;

determining, from the global feature map, a global feature corresponding to each of the set of sampling points based on a geometric association;

performing encoding on geometric information related to each of the set of sampling points, so as to generate geometric encoded information; and

training the model at least based on the global feature and the geometric encoded information,

wherein the training the model comprises:

inputting the global feature and the corresponding geometric encoded information into the model, so as to determine a geometric relationship between each of the set of sampling points and a surface of the target object;

calculating a determination error of each of the set of sampling points;

calculating a global determination error of the model; and

updating a parameter of the model based on whether the global determination error meets a precision requirement.

2 . The method according to claim 1 , wherein the training the model further comprises:

selecting a local area according to the determination error of each of the set of sampling points, and performing intensive training on the local area.

3 . The method according to claim 2 , wherein the performing intensive training on the local area comprises:

performing local feature extraction on a local sub-image in the image that corresponds to the local area, so as to obtain a local feature map;

determining, from the local feature map, a local feature corresponding to a specific sampling point in the local area based on the geometric association; and

intensively training the model using the local feature and corresponding geometric encoded information.

4 . The method according to claim 2 , wherein the selecting the local area comprises:

ranking the set of sampling points according to magnitudes of the determination error; and

determining at least part of areas where subsets of sampling points with relatively larger determination errors are located as the local area.

5 . The method according to claim 1 , wherein the training the model further comprises:

training a depth information extractor for extracting depth information from the global feature.

6 . The method according to claim 5 , wherein the training the depth information extractor comprises:

inputting the global feature into the depth information extractor, so as to obtain a fitting depth map;

comparing an actual depth map with the fitting depth map, so as to obtain a depth error; and

updating a parameter of the depth information extractor based on whether the depth error meets a precision requirement.

7 . The method according to claim 1 , wherein the generating the initial voxel envelope of the target object comprises:

generating a visual hull of the target object; and

applying a constraint condition to the visual hull, so as to determine or accurize the initial voxel envelope of the target object.

8 . The method according to claim 1 , wherein the performing random sampling on points inside the initial voxel envelope comprises:

determining a specific range in the image that corresponds to a specific part of the target object based on image recognition; and

performing intensive random sampling on points in a specific area corresponding to the specific range.

9 . A three-dimensional reconstruction method, comprising:

generating an initial voxel envelope of a target object based on an image that is obtained by photographing the target object at a plurality of angles of view;

performing random sampling on points inside the initial voxel envelope, so as to obtain a set of sampling points;

performing global feature extraction on the image, so as to obtain a global feature map;

determining, from the global feature map, a global feature corresponding to each of the set of sampling points based on a geometric association;

performing encoding on geometric information related to each of the set of sampling points, so as to generate geometric encoded information; and

inputting the global feature and the corresponding geometric encoded information into a three-dimensional reconstruction model, so as to determine a geometric relationship between each of the set of sampling points and a surface of the target object,

wherein the three-dimensional reconstruction model is trained by calculating a determination error of each of the set of sampling points, calculating a global determination error of the three-dimensional reconstruction model, and updating a parameter of the three-dimensional reconstruction model based on whether the global determination error meets a precision requirement.

10 . The method according to claim 9 , further comprising:

selecting a local fuzzy area according to a confidence level of a result of the determination, and performing fine three-dimensional reconstruction on the local fuzzy area.

11 . The method according to claim 10 , wherein the performing fine three-dimensional reconstruction on the local fuzzy area comprises:

performing local feature extraction on a local sub-image in the image that corresponds to the local fuzzy area, so as to obtain a local feature map;

determining, from the local feature map, a local feature corresponding to a specific sampling point in the local fuzzy area based on the geometric association; and

inputting the local feature and corresponding geometric encoded information into the three-dimensional reconstruction model, so as to re-determine a geometric relationship between the specific sampling point in the local fuzzy area and the surface of the target object.

12 . The method according to claim 9 , further comprising:

performing transparentization processing on part of voxels in a three-dimensionally reconstructed target voxel envelope.

13 . The method according to claim 12 , wherein the performing transparentization processing on part of voxels in the three-dimensionally reconstructed target voxel envelope comprises:

obtaining transparency of a transparent pixel in the image;

solving a voxel corresponding to the transparent pixel; and

setting transparency of the voxel corresponding to the transparent pixel based on the transparency of the transparent pixel.

14 . A three-dimensional reconstruction system, comprising:

a processor; and

a memory configured to store executable instructions,

wherein the processor is configured to read the executable instructions from the memory, and execute the executable instructions to perform:

generating process of a first initial voxel envelope of a target object based on a first image that is obtained by photographing the target object at a plurality of angles of view;

random sampling process on points inside the first initial voxel envelope, so as to obtain a first set of sampling points;

global feature extraction process on the first image, so as to obtain a first global feature map;

determination process of a first global feature, from the first global feature map, corresponding to each of the first set of sampling points based on a first geometric association;

encoding process on geometric information related to each of the first set of sampling points, so as to generate first geometric encoded information;

training process of a three-dimensional reconstruction model based on the first global feature and the first geometric encoded information;

generating process of a second initial voxel envelope of the target object based on a second image that is obtained by photographing the target object at the plurality of angles of view;

random sampling process on points inside the second initial voxel envelope, so as to obtain a second set of sampling points;

global feature extraction process on the second image, so as to obtain a second global feature map;

determining process of a second global feature, from the second global feature map, corresponding to each of the second set of sampling points based on a second geometric association;

encoding process on geometric information related to each of the second set of sampling points, so as to generate second geometric encoded information; and

inputting process of the second global feature and the second geometric encoded information into the three-dimensional reconstruction model, so as to determine a geometric relationship between each of the second set of sampling points and a surface of the target object,

wherein the training process of the three-dimensional reconstruction model comprises:

calculating process of a determination error of each of the first set of sampling points;

calculating process of a global determination error of the three-dimensional reconstruction model; and

updating a parameter of the three-dimensional reconstruction model based on whether the global determination error meets a precision requirement.

15 . The system according to claim 14 , wherein the processor is further configured to perform transparentization processing on part of voxels in a three-dimensionally reconstructed target voxel envelope.