Rendering a dental model in an image

1. A method for realistic visualization of a virtual dental model ( 1 ) in a first image ( 20 ) of a face comprising at least an inner mouth region of the face, the inner mouth region comprising teeth ( 31 ) and a mouth cavity background ( 35 ), the method comprising:

using an inpainting algorithm

to compute, based on background pixels corresponding to the mouth cavity background ( 35 ), plausible background pixels for at least a part of tooth pixels of the first image ( 20 ), the tooth pixels corresponding to the teeth ( 31 ), and

to replace at least a part of the tooth pixels by plausible background pixels, at least partially removing the teeth ( 31 ) from the first image ( 20 ); and

visualizing the first image ( 20 ) with the at least partially removed teeth ( 31 ′), wherein the first image ( 20 ) is overlaid with a representation ( 21 ) of the dental model ( 1 ),

wherein a recognition algorithm is used to determine in a previously captured image of the same face background pixels corresponding to the mouth cavity background ( 35 ), in particular wherein more of the mouth cavity background ( 35 ) is visible in the previously captured image than in the first image ( 20 ), particularly wherein the mouth cavity background ( 35 ) comprises at least a tongue and/or a palate.

2. The method according to claim 1 ,

characterized by

using the recognition algorithm to determine in the first image ( 20 ) and tooth pixels corresponding to the teeth ( 31 ).

3. The method according to claim 1 ,

characterized in that

the virtual dental model ( 1 )

is an upper jaw dental model, wherein the method is performed only for the upper jaw, only tooth pixels corresponding to the teeth ( 31 ) of the upper jaw being determined and replaced; or

is a lower jaw dental model, wherein the method is performed only for the lower jaw, only tooth pixels corresponding to the teeth ( 31 ) of the lower jaw being determined and replaced; or

comprises an upper jaw dental model and a lower jaw dental model, wherein the determining and replacing tooth pixels corresponding to the teeth ( 31 ) of the upper jaw and determining and replacing tooth pixels corresponding to the teeth ( 31 ) of the lower jaw are performed subsequently.

4. The method according to claim 1 ,

characterized in that

the image ( 20 ) is part of an image stream, and the method is performed in real time for at least a multitude of images ( 20 ) of the image stream.

5. Method according to claim 4 ,

characterized in that

the method further comprises capturing the image stream by means of a camera ( 4 ) and visualizing the dental model ( 1 ) and the two-dimensional image ( 20 ) on a displaying device ( 12 ) to a user, particularly wherein the face is the face of the user,

in particular wherein the camera ( 4 ) and the displaying device ( 12 ) are part of the same mobile device ( 10 ), wherein the method is performed by means of one or more algorithms installed on a computing unit of the mobile device ( 10 ).

6. The method according to claim 1 ,

characterized in that

the virtual dental model ( 1 ) is in a polygon mesh format and comprises a plurality of vertices.

7. A mobile device ( 10 ) comprising a camera ( 4 ) and a display ( 12 ) that arranged so that images ( 20 ) of a user's face are capturable by the camera ( 4 ) while the user watches the display ( 12 ),

characterized in that

the device comprises a computing unit with at least one algorithm that is adapted to perform the method of claim 1 .

8. A computer programme product comprising programme code which is stored on a non-transitory machine-readable medium, the machine-readable medium comprising computer instructions executable by a processor, which computer instructions cause the processor to perform the method according to claim 1 .

9. The computer programme product according to claim 8 executed on a mobile device comprising a camera ( 4 ) and a display ( 12 ) that is arranged so that images ( 20 ) of a user's face are capturable by the camera ( 4 ) while the user watches the display ( 12 ).

10. A method for realistic visualization of a virtual dental model ( 1 ) in a first image ( 20 ) of a face comprising at least an inner mouth region of the face, the inner mouth region comprising teeth ( 31 ) and a mouth cavity background ( 35 ), the method comprising:

using an inpainting algorithm

to compute, based on background pixels corresponding to the mouth cavity background ( 35 ), plausible background pixels for at least a part of tooth pixels of the first image ( 20 ), the tooth pixels corresponding to the teeth ( 31 ), and

to replace at least a part of the tooth pixels by plausible background pixels, at least partially removing the teeth ( 31 ) from the first image ( 20 ); and

visualizing the first image ( 20 ) with the at least partially removed teeth ( 31 ′), wherein the first image ( 20 ) is overlaid with a representation ( 21 ) of the dental model ( 1 ),

wherein computing the plausible background pixels comprises a stretching of a texture of the mouth cavity background ( 35 ) along a vertical direction of the face in image space, particularly wherein the vertical direction is a direction of nose to chin or a direction of chin to nose.

11. The method according to claim 10 ,

characterized in that

a three-dimensional proxy geometry is used for the stretching of the texture of the mouth cavity background ( 35 ), wherein the proxy geometry at least roughly follows an arch form slightly extending beyond the teeth ( 31 ); and/or

a fragment shader, particularly using a texel lookup downwards with a weighted vector length, is used for the stretching of the texture of the mouth cavity background ( 35 ).

12. The method according to claim 10 ,

characterized in that

a distance in the image ( 20 ) between teeth ( 31 ) of the lower jaw and the upper jaw is determined, wherein the stretching of the texture of the mouth cavity background ( 35 ) is performed only if the distance lies above a predetermined threshold.

13. The method according to claim 10 ,

characterized in that

a necessary stretching degree is determined, the necessary stretching degree being that intensity of stretching of the texture of the mouth cavity background ( 35 ) that is necessary to replace all tooth pixels that would be visible when the image ( 20 ) is overlaid with the dental model ( 1 ); and

the texture of the mouth cavity background ( 35 ) is stretched at least with the determined necessary stretching degree, in particular exactly with the determined necessary stretching degree.

14. A method for realistic visualization of a virtual dental model ( 1 ) in a first image ( 20 ) of a face comprising at least an inner mouth region of the face, the inner mouth region comprising teeth ( 31 ) and a mouth cavity background ( 35 ), the method comprising:

using an inpainting algorithm

to compute, based on background pixels corresponding to the mouth cavity background ( 35 ), plausible background pixels for at least a part of tooth pixels of the first image ( 20 ), the tooth pixels corresponding to the teeth ( 31 ), and

to replace at least a part of the tooth pixels by plausible background pixels, at least partially removing the teeth ( 31 ) from the first image ( 20 ); and

visualizing the first image ( 20 ) with the at least partially removed teeth ( 31 ′), wherein the first image ( 20 ) is overlaid with a representation ( 21 ) of the dental model ( 1 ),

wherein, before using the inpainting algorithm, it is determined whether tooth pixels are visible when the image ( 20 ) is overlaid with the dental model ( 1 ),

particularly wherein the visibility is determined only for those tooth pixels that correspond to teeth ( 31 ) of a jaw at which teeth of the virtual dental model ( 1 ) are positioned.