Segmentation for holographic images
Provided herein are methods, systems, and computer program products for calculating a refined segmentation of an image which identifies a foreground of the image and a background of the image. Embodiments include creating a refined segmentation of an image by accessing an image that includes a foreground image and background information; creating a first segmentation of the image by applying an image subtraction algorithm; creating a second segmentation of the image by applying a semantic segmentation algorithm; creating an initialization segmentation by combining the first segmentation and the second segmentation; and creating a resulting refined segmentation by refining the initialization segmentation using at least one of statistical analysis or graphical methods. Statistical analysis includes applying a Gaussian mixture model and graphical methods include a graph cut global optimization.
1. In a computing environment, a method for calculating a refined segmentation of an image which identifies a foreground of the image and a background of the image, the method comprising:
accessing an image, the image including a foreground image and background information;
creating a first segmentation of the image by applying an image subtraction algorithm;
creating a second segmentation of the image by applying a semantic segmentation algorithm;
creating an initialization segmentation by combining the first segmentation and the second segmentation; and
creating a resulting refined segmentation by refining the initialization segmentation using at least statistical analysis and graphical methods, the resulting refined segmentation being a more accurate segmentation of the image into foreground and background portions than the first segmentation and second segmentation of the image.
2. The method of claim 1 , wherein the first segmentation of the image as an array or a matrix of integers, each integer representing a pixel in the image and the value of each integer represents a likelihood of the pixel being part of the foreground of the image.
3. The method of claim 1 , wherein the second segmentation of the image as an array or a matrix of integers, each integer representing a pixel in the image and the value of each integer representing a likelihood of the pixel being part of the foreground of the image.
4. The method of claim 1 , wherein combining includes comparing corresponding integer representations of pixels from each of the first and second segmentations and resolving conflicts.
5. The method of claim 1 , wherein combining includes comparing integer representations of corresponding pixels from each of the first and second segmentations, resolving conflicts, and the initialization segmentation comprises values selected from 0, 128, and 255, with 0 representing a background pixel, 255 representing a foreground pixel, and 128 representing an unknown pixel.
6. The method of claim 1 , wherein the resulting refined segmentation is used to extract a target image and the extracted target image is one of a plurality of images which are composed to create a mesh representing a 3D volume in a hologram.
7. The method of claim 1 , wherein the statistical analysis comprises applying a Gaussian mixture model.
8. The method of claim 1 , wherein the graphical methods comprise a graph cut global optimization.
9. The method of claim 1 , wherein refining the initialization segmentation comprises applying a Gaussian mixture model to the initialization segmentation and comprises performing a graph cut global optimization to the result of applying a Gaussian mixture model to the initialization segmentation.
10. The method of claim 1 , further comprising applying the resulting refined segmentation to the accessed image to extract the foreground image.
11. A system for calculating a refined segmentation of an image which identifies a foreground of the image and a background of the image, the system comprising:
one or more computer processors; and
one or more data storage devices having stored thereon computer-executable instructions which, when executed upon the one or more computer processors, enable the system to perform:
accessing an image, the image including a foreground image and background information;
creating a first segmentation of the image by applying an image subtraction algorithm;
creating a second segmentation of the image by applying a semantic segmentation algorithm;
creating an initialization segmentation by combining the first segmentation and the second segmentation; and
creating a resulting refined segmentation by refining the initialization segmentation using at least statistical analysis and graphical methods, the resulting refined segmentation being a more accurate segmentation of the image into foreground and background portions than the first segmentation and second segmentation of the image.
12. The system of claim 11 , wherein the system comprises additional computer-executable instructions such that the system is further enabled to extract the foreground image from the image by using the refined segmentation.
13. The system of claim 11 , wherein combining includes comparing integer representations of corresponding pixels from each of the first and second segmentations, resolving conflicts, and the initialization segmentation comprises values selected from 0, 128, and 255, with 0 representing a background pixel, 255 representing a foreground pixel, and 128 representing an unknown pixel.
14. The system of claim 11 , wherein the resulting refined segmentation is used to extract a target image and the extracted target image is one of a plurality of images which are composed to create a mesh representing a 3D volume in a hologram.
15. The system of claim 11 , wherein refining the initialization segmentation comprises applying a Gaussian mixture model to the initialization segmentation and comprises performing a graph cut global optimization to the result of applying a Gaussian mixture model to the initialization segmentation.
16. A computer program product for calculating a refined segmentation of an image which identifies a foreground of the image and a background of the image, the computer program product comprising one or more hardware data storage devices having stored thereon computer-executable instructions which, when executed upon one or more computer processors of a computer system, enable the system to calculating a refined segmentation of an image which identifies a foreground of the image and a background of the image by:
accessing an image, the image including a foreground image and background information;
creating a first segmentation of the image by applying an image subtraction algorithm;
creating a second segmentation of the image by applying a semantic segmentation algorithm;
creating an initialization segmentation by combining the first segmentation and the second segmentation; and
creating a resulting refined segmentation by refining the initialization segmentation using at least statistical analysis and graphical methods, the resulting refined segmentation being a more accurate segmentation of the image into foreground and background portions than the first segmentation and second segmentation of the image.
17. The computer program product of claim 16 , comprising additional computer-executable instructions which, when executed, enable the system to extract the foreground image from the image by using the refined segmentation.
18. The computer program product of claim 16 , wherein combining includes comparing integer representations of corresponding pixels from each of the first and second segmentations, resolving conflicts, and the initialization segmentation comprises values selected from 0, 128, and 255, with 0 representing a background pixel, 255 representing a foreground pixel, and 128 representing an unknown pixel.
19. The computer program product of claim 16 , wherein the resulting refined segmentation is used to extract a target image and the extracted target image is one of a plurality of images which are composed to create a mesh representing a 3D volume in a hologram.
20. The computer program product of claim 16 , wherein refining the initialization segmentation comprises applying a Gaussian mixture model to the initialization segmentation and comprises performing a graph cut global optimization to the result of applying a Gaussian mixture model to the initialization segmentation.