Automated method for generating prothesis from three dimensional scan data, generator generating prosthesis from three dimensional scan data and computer readable medium having program for performing the method
An automated method for generating a prosthesis from a 3D scan data, the method includes extracting prep information of a prepared tooth from the 3D scan data, generating a two dimensional (“2D”) projection images by projecting the 3D scan data based on the prep information and generating a 3D prosthesis based on the 2D projection images using a generative adversarial network including a 2D encoder and a 3D decoder.
1 . An automated method for generating a prosthesis from a three dimensional (“3D”) scan data, the method comprising:
extracting prep information of a prepared tooth from the 3D scan data;
generating a two dimensional (“2D”) projection images by projecting the 3D scan data based on the prep information;
generating a 3D prosthesis based on the 2D projection images using a generative adversarial network including a 2D encoder and a 3D decoder; and
generating a prosthesis answer data used for training the generative adversarial network,
wherein generating the prosthesis answer data comprises:
converting a first answer data corresponding to the prepared tooth into a second answer data having fixed connections using a deformable registration.
2 . The method of claim 1 , further comprising:
extracting a margin line of the prepared tooth,
wherein the prep information is extracted using a prepared mesh data extracted using the margin line.
3 . The method of claim 2 , wherein the prep information includes a position of the prepared tooth, and
wherein the position of the prepared tooth is a center of gravity of the prepared mesh data.
4 . The method of claim 2 , wherein the prep information includes a position of the prepared tooth, and
wherein the position of the prepared tooth is a center of the margin line.
5 . The method of claim 2 , wherein the prep information includes a direction of the prepared tooth, and
wherein the direction of the prepared tooth is determined using normal vectors of surfaces of the prepared mesh data.
6 . The method of claim 5 , wherein when the direction of the prepared tooth is d, a number of the surfaces of the prepared mesh data is N, the normal vectors are {n 1 , . . . , n N }, x opt is a direction in which a normal vector of a point of the prepared mesh data is not obscured and T is a transpose function switching row and column indices of a matrix,
d
=
arg
min
x
opt
∈
R
3
∑
i
=
1
N
(
1
-
x
opt
T
n
i
)
is satisfied.
7 . The method of claim 1 , wherein the prep information includes a position of the prepared tooth and a direction of the prepared tooth,
wherein the 2D projection images are generated using projection planes, and
wherein the projection planes are spaced apart from a predetermined distance from the position of the prepared tooth and defined such that an opposite tooth of the prepared tooth or an adjacent tooth of the prepared tooth is visible.
8 . The method of claim 7 , wherein pixel values of the 2D projection images are defined as distances to the closest points hitting the 3D scan data when rays are emitted from the projection planes in directions of normal vectors of the projection planes.
9 . The method of claim 1 , wherein the 2D encoder is configured to receive the 2D projection images and to output a latent vector.
10 . The method of claim 9 , wherein the 3D decoder is configured to receive the latent vector and to generate coordinates of points forming the 3D prosthesis.
11 . The method of claim 1 , wherein generating a prosthesis answer data comprises:
dividing a cube-shaped initial model into eight parts and transforming the initial model to be closer to a shape of the first answer data to generate the second answer data.
12 . The method of claim 1 , further comprising:
training the generative adversarial network,
wherein training the generative adversarial network comprises:
a first training stage in which the prosthesis answer data is inputted to a 3D encoder to generate a latent vector and the latent vector is inputted to the 3D decoder to restore the prosthesis answer data.
13 . The method of claim 11 , wherein the training the generative adversarial network further comprises:
a second training stage in which a training 3D prosthesis is generated by a generator and whether the training 3D prosthesis is true or fake is determined by a discriminator.
14 . The method of claim 13 , wherein the generator comprises the 2D encoder and the 3D decoder which is trained in the first training stage, and
wherein the discriminator comprises the 3D encoder in the first training stage.
15 . The method of claim 13 , wherein a loss representing a distance difference between points of an answer mesh data and points of a predicted mesh data is used as a training objective function,
wherein a number of the points of the answer mesh data is equal to a number of the points of the predicted mesh data, and
wherein when the loss is L, the number of the points of the answer mesh data is X, the points of the answer mesh data are
{
p
gt
1
,
…
,
p
gt
X
}
and the points of the predicted mesh data are
{
p
pred
1
,
…
,
p
pred
X
}
,
L
(
p
pred
,
p
gt
)
=
∑
i
=
1
X
p
pred
i
-
p
gt
i
is satisfied.
16 . The method of claim 12 , wherein the first answer data corresponding to the prepared tooth is converted into the second answer data having fixed connections using the deformable registration, and
wherein the prosthesis answer data is the second answer data.
17 . A computing apparatus for generating a prosthesis from a three dimensional (“3D”) scan data, the computing apparatus comprising:
at least one hardware processor; and
a non-transitory computer-readable storage medium storing program instructions that, when executed by the at least one hardware processor, cause the computing apparatus to:
receive two dimensional (“2D”) projection images of a prepared tooth generated from the 3D scan data;
generate a latent vector based on the 2D projection images; and
generate coordinates of points forming a 3D prosthesis for the prepared tooth based on the latent vector,
wherein generating the latent vector and generating the coordinates comprise executing a trained generative adversarial network that includes an image encoder and a mesh decoder, and
wherein a prosthesis answer data used for training a generative adversarial network is generated by converting a first answer data corresponding to the prepared tooth into a second answer data having fixed connections using a deformable registration.
18 . A non-transitory computer-readable storage medium having stored thereon program instructions, the program instructions executable by at least one hardware processor to:
extract prep information of a prepared tooth from a three dimensional (“3D”) scan data;
generate a two dimensional (“2D”) projection images by projecting the 3D scan data based on the prep information;
generate a 3D prosthesis based on the 2D projection images using a generative adversarial network including a 2D encoder and a 3D decoder; and
generate a prosthesis answer data used for training the generative adversarial network, by converting a first answer data corresponding to the prepared tooth into a second answer data having fixed connections using a deformable registration.